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ABSTRACT 



In September 1998, the National Center for Education 
Statistics (NCES) established a 1-year task force to review the NCES ' s role 
concerning lifelong learning. The eight-member task force established a 
working definition of lifelong learning ("a process or system through which 
individuals are able and willing to learn at all stages of life, from 
preschool through old age") and conducted the following activities: (1) 

summarized and prioritized policy issues concerning lifelong learning; (2) 
synthesized exiting data to address monitoring and policy needs; (3) 
identified and prioritized gaps in existing data; and (4) developed 
recommendations on data collection strategies. The recommendations focused on 
the following lifelong learning issue areas: the adult population; learning 
attitudes and skills of adults; labor market demand for adult learning; 
participation levels and patterns; goals, incentives, and disincentives; 
investments in adult learning; adult learning providers; instructional 
delivery and new technologies; informal learning; services and accommodations 
for adults; outcomes and effectiveness; and the government's role in adult 
learning. The task force concluded that adult learning is an important area 
of education that should have a coherent data collection and reporting system 
within NCES and that NCES should take the following steps to develop such a 
system: (1) develop a compendium report summarizing existing information on 
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lifelong learning; and (2) modify existing survey instruments that collect 
relevant information. (MN) 
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SECTION 1 : BACKGROUND AND OVERVIEW 



Background 

The United States can no longer rely on the 
completion of compulsory schooling to create a 
competitive labor force and an informed citizenry. 
The competition created by a global economy, 
the proliferation of computer technologies, and 
the growth of decentralized work organizations 
all increase the need for a workforce with higher 
levels of initial skills and a greater ability for con- 
tinuous learning. In addition, the growing avail- 
ability and complexity of information and choices 
that adults face (e.g., in science and health care, 
financial services, and environmental concerns) 
make continuous learning increasingly important 
for effective functioning within society. Finally, 
population demographics — a growing cohort of 
older Americans, significantly more well educated 
than previous generations, and with ample dis- 
posable income — are creating new demands for 
adult learning opportunities. 

The growing importance of lifelong learning is 
reflected in a number of recent research and 
policy efforts. Since the 1970s, there have been 
three separate national commissions examining 
this topic. The Administration has passed a 
series of policy initiatives designed to increase 



adults’ participation in learning, including the 
Hope Scholarships, Lifelong Learning tax cred- 
its, and (newly proposed) the Learning Anytime 
Anywhere Partnership grants. In 1 998, Vice Presi- 
dent Gore sponsored a White House Summit on 
workforce skill development. The OECD has initi- 
ated efforts to develop international indicators of 
participation in education and training, and is 
considering future indicator work on lifelong learn- 
ing. 

In recognition of these shifting priorities, the Com- 
missioner of NCES established, in September 
1 998, a one-year task force to review NCES’ role 
concerning lifelong learning. The goal of the task 
force was to assist NCES management with long- 
term strategic planning — more specifically, to help 
the Center determine whether and how it should 
develop a data collection and reporting system 
on lifelong learning. This system would be de- 
signed to both monitor lifelong learning, and to 
provide data relevant to policy issues concerning 
lifelong learning. With this goal in mind, the task 
force included six NCES staff members, repre- 
senting various NCES data collection efforts that 
contain information relevant to lifelong learning 
(see table 1.1). 



Table 1.1 — Lifelong Learning Task Force Members 

Staff Member Data Collection Responsibility 



Marilyn Binkley 


ILSS 


Lisa Hudson 


Data on Vocational Education (DOVE) system 


Paula Knepper 


NPSAS, BPS, and B&B 


Andrew Kolstad 


NALS and NAAL 


Peter Stowe 


NHES/AE 


John Wirt 


Condition of Education 


Ex Officio NHES Representative 

Chris Chapman 


NHES 


ESSI Support 

Sean Creighton 


NHES/AE, ACS 
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This report summarizes the work of the Lifelong 
Learning Task Force. This section provides an 
overview of the task force’s activities, findings, 
and recommendations. Section 2 discusses in 
more detail the task force’s findings concerning 
data needs, and section 3 lists our recommenda- 
tions. The appendices provide further documen- 
tation of task force work. 

Task Force Activities 

To achieve its goal, the task force first developed 
a working definition of lifelong learning, then es- 
tablished a series of objectives. 

Definition of Lifelong Learning. At its broadest 
and most theoretical level, lifelong learning re- 
fers to a process or system through which indi- 
viduals are able and willing to learn at all stages 
of life, from preschool years through old age; this 
definition is reflected in the National Goals Panel 
objective to “build a nation of learners.” While we 
endorse this underlying conceptual definition of 
lifelong learning, this definition does not apply well 
to the task force’s charge. Given the extensive 
data collection that already exists within the Cen- 
ter on early childhood education, elementary and 
secondary education, and postsecondary educa- 
tion, we restricted our definition of lifelong learn- 
ing to adult learning. Appendix A in Volume II pro- 
vides further details on our adopted definition; as 
explained there, this definition may include some 
or all individuals who participate in postsecondary 
education, depending on the nature and/or tim- 
ing of their participation. 

We also distinguished between formal and infor- 
mal learning, and between work-related and 
avocational learning. Our definitions follow closely 
those used in the literature and in existing sur- 
veys. Formal learning is planned, mediated by an 
instructor, and includes a structured curriculum; 
it also may have an evaluation process. Informal 
learning does not have a structured curriculum 
or evaluation process; it may or may not be medi- 
ated by an instructor. Learning that is engaged in 
primarily to acquire skills or knowledge for a job 
is defined as work-related, otherwise it is 
avocational. Formal and work-related education 
received the greatest attention from our task force, 
as they were found to be the areas of greatest 
policy concern and leverage. 



With a working definition of lifelong learning in 
hand, the task force established four main objec- 
tives: 

1. Summarize and prioritize policy issues con- 
cerning lifelong learning; 

2. Synthesize existing data to address monitor- 
ing and policy needs; 

3. Identify and prioritize gaps in existing data; and 

4. Develop recommendations on data collection 
strategies. 

The activities undertaken to meet these objec- 
tives are discussed below. 

Issues Summary. To develop and prioritize the 
policy issues related to lifelong learning, the task 
force divided this area of learning into six broad 
issue areas, 1 each of which was assigned to one 
or more task force members. Each task force 
member reviewed the literature within his or her 
assigned area. Each of these reviews was supple- 
mented by a commissioned paper from an out- 
side expert, and by a general review of lifelong 
learning conducted by ESSI staff (included as 
appendixes B and C in Volume II). Table 1 .2 lists 
these policy areas and the staff assigned to them. 

The information from these various sources was 
combined into a background report that provides 
a comprehensive listing of issues relevant to 
monitor lifelong learning and to address policy 
concerns. The task force then convened a tech- 
nical review panel (TRP) of outside and Agency 
experts on adult learning and postsecondary edu- 
cation. The TRP met for one day in February 1 999 
to review the background report and help priori- 
tize the issues listed within it. Appendix D in Vol- 
ume II includes the TRP background report, list- 
ing of TRP members, and summary of the TRP 
meeting. 

Prioritizing Issues. After the TRP meeting, task 
force members divided the full list of issues in- 
cluded in the background report into high, 
medium, and low priority issues. Feedback from 
the TRP meeting contributed to this prioritization 
process, which was guided by the following crite- 
ria: each issue should (1 ) be measurable through 
national surveys, (2) be of relevance to NCES’ 
mission, (3) be of relevance to policymakers, and 
(4) affect a relatively large number of individuals 
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Table 1.2— Original Lifelong Learning Issue Areas and Assignments 



Social, Demographic, and Economic Trends 

Task Force Staff: Marilyn Binkley 

Outside Expert: David Bills, Department of Education, University of Iowa 

Acquisition of Basic Skills 

Task Force Staff: Andrew Kolstad 

Outside Expert: Mark Kutner, American Institutes for Research 

Service Delivery in Postsecondary Institutions 

Task Force Staff: Paula Knepper 

Outside Expert: David Powers, Nebraska Coordinating Commission for Postsecondary 
Education 

The Education of Workers 

Task Force Staff: John Wirt and Peter Stowe 

Outside Expert: Bonnie Nelsen, Graduate School of Management, Cornell University 

Education Technologies 

Task Force Staff: Sean Creighton, ESSI 
Outside Expert: Tom Edgerton, Sun Microsystems 

Informal and Avocational Learning 

Task Force Staff: Lisa Hudson 

Outside Expert: Barbara Butler, National Science Foundation (retired) 



or involve significant economic costs. In the 
process of prioritizing issues, the task force reor- 
ganized the issues into more focused topic ar- 
eas, listed in table 1.3 and discussed further be- 
low. The complete list of prioritized issues, clas- 
sified into the areas listed in table 1 .3, is included 
in appendix E in Volume II. (High-priority issues 
are discussed further in section 2 of this report.) 

Synthesis of Existing Data. The task force’s next 
step was to determine whether each high-prior- 
ity issue is fully addressed by existing national 
data sources. The data sources considered in this 
step are listed in appendix F in Volume II; they 
include data sets administered by NCES, as well 
as by OERI, the Department of Labor, and the 
Bureau of the Census within the Department of 
Commerce. 

While the task force went through the process of 
identifying policy issues and the data available to 
address them, another type of issue arose. For 
some of the data that should be included in a 
lifelong learning data collection system, there are 
fundamental questions about how to define un- 
derlying concepts, or how to collect data in a fea- 
sible, valid, or reliable manner. We call these 
issues conceptual and measurement issues ; a 



summary of these issues is listed in appendix G 
in Volume II. 

Identification and Prioritization of Data Gaps. 
Having identified available data sources and their 
contents, the task force then held a series of 
meetings during which we identified gaps in the 
existing data, the gaps that are most critical to 
fill, and the most appropriate procedures for fill- 
ing critical data gaps. 

Development of Recommendations. Finally, the 
task force concluded that NCES should proceed 
with the development of a data collection and re- 
porting system on lifelong learning. To meet this 
goal, two inadequacies in current data efforts 
need to be addressed. First, existing data on life- 
long learning have been under-utilized. A more 
systematic and coordinated analysis of currently 
available data is necessary to develop a lifelong 
learning reporting system. Second, although ex- 
isting data are fairly extensive, significant gaps in 
these data restrict their ability to fully describe 
adult learning. Filling these gaps is necessary in 
order to develop a comprehensive data collec- 
tion system. We developed a series of recommen- 
dations on how an effective data collection and 
reporting system should be developed, both in 
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Table 1.3— Final 12 Lifelong Learning Issue Areas 



The Adult Population 


Adult Learning Providers 


Learning Attitudes and Skills of Adults 


Instructional Delivery and New Technologies 


Labor Market Demand for Adult Learning 


Informal Learning 


Participation Levels and Patterns 


Services and Accommodations for Adults 


Goals, Incentives, and Disincentives 


Outcomes and Effectiveness 


Investments in Adult Learning 


Government Role in Adult Learning 



terms of the tasks to be pursued and the organi- 
zational structure to support this effort. We at- 
tempted to offer strategies that would proceed 
from smaller steps that meet more immediate 
needs to larger, more extensive steps that would 
realign the Center within a “lifelong learning” 
framework. 

Summary of Findings and 
Recommendations 

This section provides a brief overview of the task 
force’s findings on the important policy areas 
within lifelong learning and the data gaps that exist 
within each area. It also provides a brief sum- 
mary of the recommendations proposed for fu- 
ture NCES work in this area. A more detailed dis- 
cussion of these issues is contained in the sub- 
sequent sections of this report. 

The Adult Population. Unlike K-12 education, 
adult learning is not compulsory; this learning is 
often voluntary, or linked to work requirements, 
which differ among occupations. As a result, par- 
ticipation can vary among groups of adults, such 
as those with different levels of formal education 
or in different jobs. It is thus informative to moni- 
tor who participates in adult learning, for general 
descriptive purposes, to understand why some 
adults participate and others do not, and to de- 
termine whether there are inequities in the distri- 
bution of learning opportunities. 

We noted that there is a great deal of information 
available in existing surveys with which to de- 
scribe adults. However, some questions are asked 
differently from one survey to another, which can 
make findings inconsistent among surveys and 
difficult to interpret. Specifically, we need to con- 
sistently distinguish those with a regular high 



school diploma from those with a GED or other 
alternative diploma. We also need to collect con- 
sistent information on disability status and more 
information on adults’ labor market status. 

Learning Attitudes and Skills of Adults. Adults 
participate in learning because they want to and / 
or need to. Thus, it is difficult to fully understand 
participation patterns without understanding the 
skills available within the adult population, their 
attitudes toward continued learning, and (ad- 
dressed in the next subsection), the skills used 
or needed in the labor market. 

Although NCES currently conducts assessments 
of adult literacy, few data are available to reveal 
adults’ perceptions of their learning needs or the 
opportunities they have to fill these needs. In ad- 
dition, we are not able to link information on the 
skills sought through lifelong learning activities 
to the skills required by jobs, or adults’ current 
skill levels; these measures of skill “supply” and 
“demand” need to be brought into alignment. 

Labor Market Demand for Adult Learning. 

Much of the learning adults engage in is related 
to work; the labor market is an important source, 
if not the most important source, of the demand 
for lifelong learning. Thus, to help understand or 
predict the nature and extent of lifelong learning, 
it is necessary to understand the skill and learn- 
ing needs that exist within the labor market. 

Data on the skills used in the labor market are 
available in the Department of Labor’s 0*NET 
system, which lists over 1 ,000 occupations and 
their job requirements; these data could be linked 
to data on the composition of the labor market in 
order to track labor market skill demands. Data 
from 0*NET or other sources are also needed to 
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monitor changes in the extent to which occupa- 
tions require continual or periodic learning; this 
factor is probably as important as changes in skills 
in general in determining the need for adult learn- 
ing. We need to learn more about the feasibility 
of 0*NET and other occupational data for these 
purposes. 

Participation Levels and Patterns. Obviously, 
one cannot monitor lifelong learning without moni- 
toring the nature and extent of participation in 
learning activities. Participation levels and pat- 
terns provide the most fundamental information 
to describe the enterprise of lifelong learning. 

Data on participation are widely available from a 
number of sources, most notably the NHES/AE. 
But there are two limitations in currently available 
data. First, the scope of activities included in cur- 
rent surveys is too narrow, often excluding "non- 
traditional” learning activities (informal learning, 
continuing education, noncredit coursetaking, 
etc.) that are believed to be areas of rapid growth. 
Second, data on the “intensity” of learning activi- 
ties (the time involved in learning) are unreliable. 
Intensity measures are important because they 
can serve as an indicator of the extent of human 
capital investment in adult learning, the amount 
of skill acquisition involved, or the opportunity 
costs of participation. 

Goals, Incentives, and Disincentives. Adult 
learning encompasses a broad array of activities 
that can be engaged in for personal, social, eco- 
nomic, or other reasons. In addition, adults typi- 
cally face a wide array of influences — at home, 
at work, and in their personal lives — that influ- 
ence the likelihood of participation in learning. 
These incentives and disincentives can be as 
strong as an employer or professional require- 
ment for skill-updating, or can involve more com- 
plicated influences such as competing demands 
on time from family and work responsibilities. In 
order to describe lifelong learning and understand 
why it does or does not occur, we need to know 
what role it plays in adults’ lives — what larger 
goals it fulfills, and what incentives and disincen- 
tives affect adults’ propensity to participate. 

These factors are currently not measured well. A 
model of the decisionmaking process involved in 
lifelong learning is needed, as well as a re- 



conceptualization of terms. At the least, surveys 
should assess in a more valid and coherent way 

(1) the domain of life to which a learning activity 
applies (e.g., one’s personal life or job situation), 

(2) the intended benefits that serve as intrinsic 
motivators to participation, (3) the inducements 
(such as requirements) that serve as extrinsic mo- 
tivators, and (4) the barriers, constraints, and 
other disincentives that inhibit participation. 

Investments in Adult Learning. Two major policy 
issues concern the costs involved in adult learn- 
ing. One issue is the extent to which employers 
are investing in employee training and skill de- 
velopment; some conditions that characterize the 
labor market (relatively high turnover rates, trans- 
portability of skills) may motivate employers to 
underinvest in these activities. The second issue 
is the extent to which the costs borne by indi- 
viduals (including opportunity costs) restrict ac- 
cess to learning opportunities, particularly for eco- 
nomically and educationally disadvantaged 
adults. 

Data currently exist on whether adult learners 
receive employer support and the amount the 
learner pays for participation. These data could 
be improved in two ways. It would be useful to 
have more comprehensive information on the na- 
ture and extent of employer support for learning 
opportunities, and on the nature (in addition to 
the amount) of the costs individuals incur when 
they participate in learning activities. A survey of 
employers would be very useful for addressing 
the former issue. 

Adult Learning Providers. One important fea- 
ture for both monitoring and policy purposes is 
who provides or sponsors learning activities for 
adults. Knowledge about providers contributes to 
our understanding of the role played by two key 
groups of policy interest, employers and postsec- 
ondary institutions. Ideally, one would collect in- 
formation on providers from surveys of the rel- 
evant provider organizations. However, for adult 
learning, providers are so varied and decentral- 
ized that it would be virtually impossible to con- 
struct a universe list of all providers. It is possible, 
however, to construct such lists for the two key 
providers of employers and postsecondary insti- 
tutions. 
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Asking adults who provides their learning oppor- 
tunities is the simplest approach to determining 
the full range of providers. This approach is taken 
by NHES/AE and other surveys. But in situations 
where different groups (such as employers and 
postsecondary institutions, or community and 
government agencies) collaborate to provide edu- 
cation or training activities, adults are unlikely to 
be able to accurately report who the provider is. 
Survey questions to collect this information should 
be redesigned to provide more valid, standard- 
ized information in a format that allows adults to 
report on collaborative arrangements (when pos- 
sible). 

Instructional Delivery and NewTechnologies. 
Another central characteristic of lifelong learning 
is the means employed to deliver instruction, par- 
ticularly the role of new technologies in instruc- 
tional delivery. These technologies are important 
because they potentially offer a means of over- 
coming many of the access problems faced by 
adults, including the constraints of time and dis- 
tance. 

The ongoing series of Distance Education PEQIS 
surveys provides the best source of information 
on this delivery method. However, the most re- 
cent version of this survey did not include ques- 
tions on the targeting of these programs to work- 
ers or other groups of adult learners. In general, 
we need more and better information on all forms 
of technology-based delivery systems and their 
role in adult learning, including the use of dis- 
tance education to serve specific groups of adult 
learners. 

Informal Learning. Informal learning is probably 
the oldest form of learning and the most com- 
mon. New technologies and other factors may be 
making this form of learning even more common, 
as learners have access to more sources of in- 
formation, and the continued specialization of 
knowledge makes it easier to acquire “chunks” of 
knowledge through less formal means. It is use- 
ful to know the extent to which and ways in which 
informal learning is either supplementing or re- 
placing formal instruction as a means of adult 
learning. 

Information on informal learning is currently re- 
stricted to work-related learning, which seems 
appropriate given policy priorities. However, this 
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information is very limited, and the questions 
asked about this topic vary among surveys. A 
broader and more consistent approach needs to 
be taken to data collection on work-related infor- 
mal instruction. 

Services and Accommodations for Adults. 

One access issue applies primarily to postsec- 
ondary institutions, where special services, offer- 
ings, or delivery methods can be used to make 
learning more appropriate or accessible for the 
adult learner. Distance education and other tech- 
nology-based delivery systems are one means 
of increasing access, but less technological ap- 
proaches can be used as well — such as contract 
training; noncredit and continuing education 
courses; special services to transport, counsel 
or otherwise serve adults; and competency-based 
and other outcomes-based assessment and 
credentialing mechanisms. 

Current data on these issues are very limited. 
Surveys of adults and of postsecondary institu- 
tions should be revised to better address these 
issues. In addition, one or more PEQIS surveys 
could be designed to specifically target the issue 
of services and accommodations provided for 
adults within postsecondary institutions. 

Outcomes and Effectiveness. Obviously, one 
of the most important issues to assess for both 
monitoring and policy purposes is the outcomes 
associated with adult learning. Individuals, the 
government, and private organizations all con- 
tribute time and money to these activities in the 
expectation that they will receive benefits from 
doing so. Of particular concern to NCES are the 
benefits received by the adult learner. 

Data to examine the labor market effects of par- 
ticipation exist in the longitudinal surveys con- 
ducted by the Bureau of Labor Statistics (BLS) 
and the Census Bureau. These data should be 
analyzed for this purpose. In addition, NCES sur- 
veys of adults should be expanded to ask about 
the types of benefits (if any) adults receive from 
their participation. 

Government Role in Adult Learning. Finally, the 
federal government has begun taking a more 
active role in encouraging participation in adult 
learning, and policymakers need to know if these 
efforts are working. To help guide future policy 
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efforts, it is also useful to know how government 
and corporate support for adult learning in the 
U.S. compares to that in other countries. 

Current survey efforts, some being conducted in 
cooperation with the OECD, are designed to ad- 
dress the issues of federal incentives and inter- 
national comparisons of government support. 
However, more reliable data on the use of fed- 
eral incentives could be provided through tax 
records. NCES should seek a cooperative agree- 
ment with the IRS to gain access to such infor- 
mation. 

Recommendations. The task force believes that 
adult learning is an important area of education 



that should have a coherent data collection and 
reporting system within NCES. We recommend 
that the Center take steps to develop such a sys- 
tem, using (and expanding) the wealth of data that 
are currently available on this topic. The first steps 
should be (1) the development of a compendium 
report summarizing currently available information 
on lifelong learning (using an indicators format), 
and (2) the modification of existing survey instru- 
ments that collect information relevant to lifelong 
learning, many of which are scheduled for 
redesign in the immediate future. A number of sub- 
sequent tasks are possible after these initial steps 
have been taken, depending on the level of 
financial resources and other support available. 
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SECTION 2: 

LIFELONG LEARNING ISSUES AND DATA GAPS 



This section of the report lists the high-priority 
issues concerning lifelong learning that were iden- 
tified by the task force, grouped into the twelve 
categories listed in table 1 .3. This section also 
summarizes, for the issues within each category, 

(1) the data available to address these issues, 2 

(2) the most important data gaps that need to be 
filled to address these issues, (3) recommended 
procedures to fill identified data gaps, and (4) data 
collection instruments that could provide sample 
questions for filling data gaps. This section pro- 
vides a comprehensive listing of data gaps; in 
section 3, we recommend which of these data 
gaps should be addressed first. 

The data collection instruments considered as 
sources to fill data gaps were those listed in ap- 
pendix E, as well as surveys of adult education 
and training conducted by the following six coun- 
tries: Canada, Finland, Germany, Sweden, Swit- 
zerland, and the United Kingdom. 3 Two surveys 
considered, the Survey of Employer-Provided 
Training (SEPT) and the National Employer Sur- 
vey (NES), are not likely to be available in the 
future.The Bureau of Labor Statistics has no plans 
to re-administerthe SEPT, and OERI has no plans 
to fund the NES after 2000. 

1. The Adult Population 

Who participates in adult learning is a fundamen- 
tal question to ask about this enterprise, particu- 
larly since participation is often voluntary. Of par- 
ticular interest are learner characteristics that 
relate to equity concerns. 

High Priority Issues 

1 a. What are the education attainment levels of 
the adult population?YJha\ is the trend in educa- 
tion attainment over time? 

1 b. What is the age composition and racial/eth- 
nic composition of the adult population, and what 
are the education attainment levels of these sec- 
tors of the population? What are the trends in 
these population characteristics? 

1c. What proportion of the adult population is 
learning disabled? 



Id. How many adults, and which adults, use com- 
puters at home or at their workplace? What are 
the trends over time in the use of computers by 
adults at home and at work? 

Data Currently Available 

Data to describe the basic demographic charac- 
teristics of adults are available in all surveys of 
this population. Education attainment data are 
also widely available, although questions on this 
topic have not been standardized. The main con- 
cern is distinguishing between high school diplo- 
mas earned in the “normal” manner and those 
earned after dropping out, such as through a GED 
test. Self-reported information on disabilities is 
available from the NALS, IALS, ILSS, NPSAS and 
its follow-ups, SIPP, CPS, and some administra- 
tions of the NHES, again using nonstandardized 
questions. It is unclear to what extent differences 
in question wording affect disability rate estimates, 
but it seems likely that there would be some ef- 
fect. 

It is obviously difficult to survey adults who do 
not speak English. The three most critical sur- 
veys for adult learning, the NHES/AE, NALS, and 
IALS, survey adults in both English and Spanish, 
which is the most feasible approach for including 
at least some non-English speakers. 

Data on the use of computers at home exist only 
in the CPS; no survey asks adults about the use 
of computers at work, although the NES asks 
employers about computer use among different 
types of employees (managers, clerical support, 
etc.). Ideally, we would like to have information 
on not just the extent and frequency of computer 
usage, but also the purposes for which comput- 
ers are used (particularly learning purposes). 

Although not listed above, it is also important to 
have information on adults’ labor market status. 
Whether an adult is employed, whether the adult 
is in a unionized job, the occupation and industry 
the adult is employed in, and the size of the adult’s 
employer are all related to the likelihood of an 
adult’s participation in learning activities. Without 
a complete listing of these labor market charac- 
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teristics, researchers cannot accurately model the 
labor market influences on participation. With the 
exception of information on union membership 
and employer size, these data are collected from 
most surveys. 

The exclusion or under-inclusion of certain groups 
is common in surveys of adults. Perhaps the major 
exclusion is those adults who do not live in civil- 
ian households, including those in the military and 
in prison. Individuals at certain life stages that may 
be linked to participation (e.g., displaced home- 
makers, displaced workers) are also often not 
identifiable. 

Most Important Data Gaps to Fill 

• Questions need to be standardized across 
data sources that describe adults’ background 
and characteristics, particularly questions on 
education attainment level and disability sta- 
tus. 

• Questions need to include a full range of la- 
bor market information, including employment 
in a unionized job and employer size, on all 
surveys that examine adult participation in 
learning activities. 

How to Fill Data Gaps 

• Review relevant surveys and modify questions 
on education attainment and disability status 
as necessary. 

• Add questions on employment in a unionized 
job and employer size to relevant surveys, in- 
cluding the NHES/AE, NAAL, ILSS, NELS, 
and NPSAS and its follow-ups. 

2. Learning Attitudes and 
Skills of Adults 

Participation in adult learning activities is medi- 
ated by adults’ skill and knowledge levels and by 
their attitudes toward learning. Knowledge of 
these adult characteristics is particularly impor- 
tant to help understand participation patterns. 

High Priority Issues 

2a. To what extent do adults recognize the need 
to be continuous learners? 

2b. What are adults’ perceptions of the value of 
lifelong learning activities for their own lives? 



2c. To what extent do adults feel they have the 
skills and knowledge they need to function effec- 
tively at their current job and for the job they would 
like to have in the future? To what extent are adults 
comfortable with their skill levels? 

2d. To what extent are adults, particularly work- 
ers, satisfied with existing opportunities to improve 
their education and skills? What changes would 
they like to see in these opportunities? 

2e. What types of skills and knowledge do adults, 
particularly workers, seek to acquire through adult 
learning? What are the trends in the types of 
knowledge/skill development received? 

2e1 . How generic or transferable is the educa- 
tion and training that workers receive? To what 
extent do workers perceive the education and 
training they receive to be useful for other 
jobs in their company, and/or for jobs in other 
companies? 

2f. What is the level of basic skills in the adult 
population? What are the trends overtime in skill 
levels? 

Data Currently Available 

With the exception of skill levels, data on these 
issues tend to be spotty. There is virtually no in- 
formation available on adults’ views on learning. 
The ILSS proposes to ask some questions re- 
lated to this issue, but these questions may not 
be included in the final version of the survey, as 
they have not been given high priority. The IALS 
asked adults their perception of the adequacy of 
their English and math skills for job opportunities 
and for daily life, but it is not clear whether these 
questions will be included on the ILSS. 

The NELS fourth follow-up is the only survey that 
asks adults directly about satisfaction with exist- 
ing opportunities to improve education and skills; 
this survey asks adults to what extent they are 
satisfied with the opportunities their job provides 
for further education and training. The IALS may 
include a question asking adults the extent to 
which they wish their employer provided more 
training opportunities. This issue is also indirectly 
addressed in questions that ask adults if there 
were courses they would have liked to have taken, 
but did not (as in the NHES/AE). 
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Little information is available on the particular skills 
that adults seek to acquire through adult learning 
activities, although the SEPT (which is not sched- 
uled to be repeated) collected such information 
for employer-provided training. Some information 
on skills sought can probably be gleaned (at con- 
siderable expense) from the course titles collected 
in the NHES/AE, IALS, and ILSS. The NLS-Y is 
the only survey that asks about skill generalizability 
or transferability. This survey asks adults if the skills 
they learned could be used at the same job with a 
different employer, at a different job with the same 
employer, or at a different job with a different em- 
ployer. 

Data on adults’ skill levels in particular literacy 
domains are available from the national and 
international adult literacy assessments; data on 
vocational aptitudes are available from the NLS-Y. 

Most Important Data Gaps to Fill 

• We need better data on adults’ perceptions of 
their learning needs and of the opportunities 
they have to fill these needs. 

• We need data on skills sought in learning ac- 
tivities that can be linked to (1 ) adults’ current 
skill levels, and (2) skills used in the labor 
market (see issue area 3). For work-related 
courses, we need at the very least to distin- 
guish between courses that focus on skills 
development versus on other types of train- 
ing (new worker orientation, health and safety, 
EEOC, etc.). 

How to Fill Data Gaps 

• Develop and add relevant questions on per- 
ceptions of skill and learning needs to relevant 
surveys (most notably NHES/AE, ILSS, and 
NAAL). 

• Potential survey items may exist in IALS, 
ILSS, and other countries’ surveys (Fin- 
land). 

• Add question(s) on perceived availability of 
learning opportunities to relevant surveys 
(most notably NHES/AE, ILSS, NAAL, and 
NLS-Y). 

• Potential survey items may exist in NELS, 
IALS, NES, and other countries’ surveys 
(Finland, Germany, United Kingdom). 



• Add questions to NHES/AE (or CPS supple- 
ment) on skills sought in learning activity that 
will allow linkage to skills used in the labor 
market (see issue area 3). 

• Potential survey items may exist in NLS-Y, 
NES, and SEPT. 

• Add questions to ILSS and NAAL on skills 
sought in learning activity that will allow link 
to skills assessed. 

• Potential survey items may exist in NALS. 

3. Labor Market 
Demand for Adult Learning 

Much of adult learning is job-related, and thus 
related to the skills used and learning demands 
within the labor market. Tracking these labor mar- 
ket skill and learning needs helps project the de- 
mand for adult learning, both overall and for adults 
in various occupations or labor market sectors. 

High Priority Issues 

3a. What skills are used in the labor market, and 
to what extent is the learning of new skills or 
knowledge characteristic of jobs in the labor mar- 
ket? What are the trends in the use of skills and 
learning, overall and within specific jobs that domi- 
nate the economy? What are the trends in use of 
specific types of skills or learning? Are skills or 
learning demands becoming more polarized over 
time? 

3a 1 . To what extent are occupations character- 
ized by continuous skill development? Do occu- 
pations that have higher initial education demands 
also have higher continuing education demands? 
What is the trend over time in the need for con- 
tinuous skill development (particularly in high-tech 
occupations)? 

3a2. What is the job composition of the labor 
market, by skill or education requirements? To 
what extent is the mix of skills required in the la- 
bor market changing— in general, and in particu- 
lar occupation or industry areas? 

3a3. What types of skills and learning are increas- 
ing and decreasing in demand? 

3a4. To what extent do workers report that the 
skill, knowledge, and learning requirements for 
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particular occupations are changing?Jo what do 
workers attribute these changes? 

3b. To what extent are the skills and knowledge 
required of workers specific to an individual em- 
ployer, occupation, or industry, or generic in na- 
ture? 

3c. To what extent do workers rely on (use) skills 
and knowledge acquired through formal educa- 
tion, rather than skills and knowledge acquired 
within the work context? 

Data Currently Available 

The DoL 0*NET system provides detailed in- 
formation on job skills, by occupation category, 
including education requirements, skill require- 
ments, and the extent to which jobs are char- 
acterized by continuous education or training. 
This data set has not been used by NCES, but 
could potentially provide a valuable source of 
information if we could link 0*NET classifica- 
tions to the occupations of adults in our data 
collections. At a much less detailed level, the 
IALS (and possibly ILSS) collects information 
on the skills adults use at work, focusing on 
skills related to those assessed in these stud- 
ies. Trend data on changes in skills used could 
be available every year in 0*NET (assuming 
0*NET could be linked to annual CPS occupa- 
tion data), and every 10 years in IALS/ILSS. 

Data on the demand for learning are also avail- 
able in limited form from the NHES/AE, which 
asks about courses taken in response to em- 
ployer requirements and whether the adult has 
professional continuing education require- 
ments; and from the NES, which asks employ- 
ers the extent to which workers in broad job- 
categories (managers, supervisors, technical 
workers, clerical staff, and frontline or produc- 
tion workers) need to continue their formal edu- 
cation. 

It might be difficult for workers to determine the 
extent to which they use skills that were ac- 
quired at work or in schooling. As one variation 
on this, the SEPT asked workers how they 
learned the skills required on their current job, 
and the most important way they acquired their 
job skills — through formal training, informal 
training, learned on own, or “already knew job 
duties.” A less direct way of determining the role 



O 




of initial education in the development of work 
skills is to ask workers or their employers how 
long it takes for employees to become fully profi- 
cient in their job. The longer this takes, the greater 
the role of on-the-job learning compared to for- 
mal education. Two surveys (neither of which will 
be available in the future) provide data on this 
topic. The SEPT asks workers how long it was 
before they were comfortable performing their job 
duties. The 1997 NES asked employers how long 
it takes the average employee to become fully 
proficient in his or her job. 

Participation in employer-sponsored learning in- 
dicates the extent to which work-based instruc- 
tion is needed to supplement initial education 
during a worker’s tenure. Likewise, participation 
in postsecondary education (for one’s current job) 
indicates the extent to which formal instruction is 
needed to supplement a worker's initial educa- 
tion. These data are available from the NHES/ 
AE. 

The 0*NET should provide information on the 
extent to which job skills are job-specific or ge- 
neric, but there are no sources of information on 
the extent to which job skills are specific to an 
employer. As mentioned above, the NLS-Y asks 
adults if the skills they learned through training 
could be used at the same job with a different 
employer, at a different job with the same em- 
ployer, or at a different job with a different em- 
ployer. To describe skills used in the labor mar- 
ket, workers could be asked a similar question 
about the skills they perform on their jobs. It is 
unclear, however, whether workers can accurately 
make such judgments. 

Most Important Data Gaps to Fill 

• We need data on the skill characteristics of 
the labor market. 

• We need data on the extent to which jobs de- 
mand continuous or periodic learning, in a 
form that can be linked to adults’ perceptions 
of their education and training needs, the 
methods adults use to fill those needs, and 
their participation rates in work-related learn- 
ing. 

• As a long-term goal, we need to align data on 
skill levels, skills sought in adult learning ac- 
tivities, and skills used in the labor market. 
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How to Fill Data Gaps 

• Requisite data on labor market skill demands 
may be available from 0*NET, but we currently 
know little about the feasibility of using this 
data system to collect such information. First, 
we must learn more about 0*NET and its ap- 
plicability for our purposes. Then, if feasible: 

• Use the 0*NET system to describe the skill 
characteristics of the labor market. 

• Use 0*NET in combination with CPS or other 
Census data (describing the job composition 
of the labor market) to monitor skills used in 
the workforce. 

• Work with DoL to find a cost-effective way to 
code occupations at the 3-digit level, permit- 
ting a detailed linking of occupations in NCES 
surveys to 0*NET (to determine, for example, 
the proportion of adults in occupations that 
use skill X who have taken training to improve 
skill X). 

4. Participation Levels and Patterns 

Probably the most fundamental data on adult 
learning are those that describe the extent to 
which adults participate in learning, both currently 
and over time. A detailed examination of this is- 
sue — linking participation rates to the character- 
istics of adults, labor market needs, incentives 
and constraints to participation, etc. — provides 
the core of a monitoring system on adult learn- 
ing. 

High Priority Issues 

4a. How many, and what proportion of adults par- 
ticipate in lifelong learning activities? In other 
words, what is the extent of participation, and what 
is the incidence rate for participation in lifelong 
learning? 

4b. How much time do adults devote to each life- 
long learning activity? In other words, what is the 
intensity of participation in lifelong learning ac- 
tivities? Do incidence and intensity rates suggest 
different levels of participation? 

4c. Which adults participate in lifelong learning 
activities and which do not? Do participation pat- 
terns vary among groups of policy interest? How 
do participation rates vary across the adult 
lifespan? 



4c1 . Which adults participate in each type of learn- 
ing — formal, informal, work-related, and 
avocational? Are informal learning activities more 
common among those who are more self-moti- 
vated learners? 

4d. What are the trends over time in participation 
in lifelong learning? Are there groups of adults 
for whom these learning activities are becoming 
more or less popular? Are participation trends 
similar for different types of adult education? 

4d1 . What are the trends in participation by adults 
in particular age-cohorts and at particular edu- 
cation attainment levels? 

4d2. What are the trends in participation for each 
type of learning (formal, informal, work-related, 
and avocational)? 

4e. How extensive are enrollments of part-time 
and adult students at postsecondary institutions ? 
What trends are occurring in these enrollments? 

4f. What are the characteristics of ABE/ASE-eli- 
gible adults who are not interested in participat- 
ing in these programs (versus those who are in- 
terested)? What are the characteristics of ESL- 
eligible adults who are not interested in partici- 
pating in these programs? 

Data Currently Available 

Basic data on participation levels and the char- 
acteristics of participants in adult learning are 
widely available. While much information can be 
gleaned from existing data, no surveys collect 
information on the full range of formal and infor- 
mal learning activities that encompass adult learn- 
ing. Most surveys focus on formal activities, and 
some (SIPP, NLS-Y) focus primarily on work-re- 
lated or employer-sponsored activities. The N ALS 
examines only adult basic education (including 
ESL). No survey provides data on participation 
in informal avocational learning activities. The 
ILSS is the only survey that (if developed as cur- 
rently planned) will include both formal learning 
activities and informal work-related activities. 

Data on (for-credit) enrollments at postsecond- 
ary institutions are also available from a large 
number of sources, all of which allow one to iden- 
tify “adults” by age. If one wants to define adults 
by delayed entry and/or part-time attendance sta- 
tus, NPSAS/BPS/B&B and NLS-Y are the only 
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available data sources. Trend data are available 
from most surveys, although some (IALS/ILSS 
and NLS-Y) have 1 0 or more years between each 
new data collection or cohort. 

Data on the amount of time spent in specific learn- 
ing activities (hours of instruction) are available 
in NHES/AE, IALS, ILSS, and SEPT. However, 
only the SEPT (which used daily training logs) is 
likely to have very reliable data on this topic. There 
are a number of issues that need to be resolved 
concerning measures of instructional “intensity” 
or “volume” (see appendix G). The most funda- 
mental issue is how to collect reliable intensity 
data when one asks about activities over a full 
year (which is typically done to avoid concerns 
about a “seasonality” bias). For example, recent 
analyses of the NHES/AE suggest that even with 
built-in range checks, overall course intensity fig- 
ures are often too large to believe. 

To determine participation rates among those who 
are eligible for basic skills education or ESL, we 
must first define the “eligible” populations. For 
adult basic education participants, those who 
have not completed high school are typically con- 
sidered the eligible population (e.g., NHES/AE 
uses this criterion). Since education level is avail- 
able in all surveys of adults, this population can 
readily be identified. However, if better informa- 
tion were available on the specific literacy levels 
required in the labor market, basic skills criteria 
could also be set based on adult literacy assess- 
ment scores; this would provide a more direct 
measure of who is in need of basic skills educa- 
tion. (At least one researcher has noted that high 
school completion should not be used to define 
the eligible population, because a few adults with 
a high school diploma or higher degree have abili- 
ties below the basic skills level.) For ESL eligibil- 
ity, the NHES/AE uses the criterion of those who 
do not speak English at home. Obviously, the limi- 
tation here is that some or all adults with limited 
English skills are often excluded from surveys. 
As previously mentioned, the NHES/AE, NALS, 
and IALS are administered in English and Span- 
ish to allow the inclusion of native Spanish-speak- 
ers. 

Most Important Data Gaps to Fill 

• We need to expand the range of learning ac- 
tivities surveyed to include informal work- 
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related activities and possibly informal 
avocational activities. 

• We need comparable questions about infor- 
mal activities on the surveys that include this 
topic. 

• We need better data on the intensity of 
coursetaking. 

How to Fill Data Gaps 

• Add questions about informal learning activi- 
ties to NHES/AE, using IALS as a model. 

• Experiment with new ways to collect data on 
intensity; modify NHES/AE accordingly. 

• Potential survey items may exist in ILSS 
and other countries’ surveys (Canada, 
Germany, Sweden, Switzerland, England). 

5. Goals, Incentives, and Disincentives 

After examining the extent of participation in adult 
learning and who participates, the next logical 
question to address is why adults choose to par- 
ticipate (or not). 

High Priority Issues 

5a. What are adults’ goals or reasons for partici- 
pating in adult learning activities? What propor- 
tion of their learning is for work-related versus 
avocational reasons? Of work-related learning, 
what proportion is taken for initial or qualifying 
training, retraining, skill upgrading, or career ad- 
vancement? 

5a1 . What are adults’ educational goals when they 
participate in adult learning? What proportion of 
adults intend to earn a degree or other creden- 
tial? What type of credential do they seek (e.g., 
postsecondary certificate, company certificate, 
state or professional license)? 

5a2. What are adults’ reasons for engaging in 
work-related learning?Wba\ proportion is under- 
taken as a proactive effort to increase earnings 
or job status versus as a reactive effort to main- 
tain a current job? What proportion is undertaken 
in response to technological or organizational 
change, or to job loss? 

5b. What are the incentives and disincentives to 
participation in lifelong learning activities that in- 
fluence the behavior of adults (including partici- 
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pants, interested non-participants, and uninter- 
ested non-participants)? To what extent is par- 
ticipation limited by a lack of incentives rather than 
by disincentives? 

5b1 . Among those interested in participating in 
adult basic education programs, what benefits are 
perceived to accrue from participation, and what 
costs or other hindrances (e.g., lack of self-confi- 
dence) outweigh those benefits? Among those 
not interested in participating, what factors con- 
tribute to the perception that programs offer no 
benefits? 

5b2. What are the motivating factors (incentives) 
among those who participate in each type of adult 
basic education program (ABE, ASE, ESL)? 

Data Currently Available 

Data on adults’ educational goals when partici- 
pating in learning are fairly straightforward and 
readily available. These data are collected in the 
NHES/AE, IALS, ILSS, and SIPP. 

In general, many surveys (NHES/AE, IALS, 
NELS, SIPP, NLS-Y) ask about learners’ “goals” 
or “reasons” for participating in a learning activ- 
ity, but the answers to these questions are typi- 
cally confused or combined with the answers to 
another question — i.e., to which domain of life 
(worklife or personal life) does a learning activity 
apply? For example, one set of responses includes 
the following: to improve basic skills, get a creden- 
tial, improve skills on job, get a new job, or per- 
sonal reasons. Moreover, each survey includes a 
different combination of “goals,” and some allow 
the learner to select only one while others allow 
more than one to be selected. Two surveys (SEPT, 
NELS) have questions about learning outcomes 
that could be reframed as goals or intended ben- 
efits; it would be interesting to ask both questions 
the same way (e.g., did you hope to get a promo- 
tion; did you get a promotion?). 

We believe that learning domains and intended 
benefits (the latter replacing the terms “goals” and 
“reasons”) should be clearly separated, or com- 
bined in a systematic way. One approach might 
be to first ask if a course was taken primarily for 
work or for personal reasons, then for work-re- 
lated courses ask if the course was taken in ref- 
erence to the adults’ (then) current job or a future 
job. After establishing those learning domains, 



more focused questions could be asked about 
intended benefits (improve basic skills, improve 
family interactions, relieve stress, get a promo- 
tion, get a higher paying job, etc., as appropri- 
ate). 

Intended benefits are one class of incentives to 
learning; these typically imply some internaliza- 
tion of the goal (i.e., the learner is self-motivated). 
Other incentives are external. For example, an 
employer’s desire that employees take a training 
course or family pressure could induce an adult 
to participate in a learning activity. Other external 
motivators are legal, professional, or other work- 
related requirements. To capture the full range of 
incentives and requirements that may affect an 
adult’s decision to participate in learning, we pre- 
fer the term inducement. In general, data on in- 
ducements to participation are insufficient. As 
mentioned above, information on intended ben- 
efits is often confused with other issues, making 
it difficult to clearly determine the learners’ goals, 
or to compare goals from one survey to another. 
Questions on external inducements are not fully 
covered in any survey. The NLS-Y and NHES/AE 
ask if a course was taken in response to an em- 
ployer requirement. The NHES/AE also asks a 
general question about whether the adult has 
continuing education requirements, but does not 
link specific coursetaking to the need to meet 
those requirements. The NELS asks those par- 
ticipating in work-related education if their train- 
ing was required or encouraged by their employer, 
but this question is also not linked to specific 
courses. 

Existing surveys tend to focus more on disincen- 
tives (“barriers” or “constraints”) to participation 
than on inducements. Questions on disincentives 
are asked in NHES/AE, IALS, and ILSS. Each 
survey lists the reasons respondents did not par- 
ticipate in courses that they wanted to take, with 
some attempt made to capture situational (e.g., 
lack of time) and institutional (e.g., entry require- 
ments) constraints. Nonetheless, these questions 
vary in format and response options and, at least 
in the case of NHES/AE, have been found to be 
unsatisfactory; one analyst refers to these items 
as providing data on “alibis.” 

The current lack of coherence and utility charac- 
terizing data on intended benefits, other induce- 
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merits, and disincentives reflects, we believe, a 
larger inability to model the decisionmaking pro- 
cess that adults undergo when considering par- 
ticipation in a learning activity. In general, data 
collection on these issues could benefit from a 
more rational approach to the decisionmaking 
process. 

Most Important Data Gaps to Fill 

• We need to develop items that clearly distin- 
guish between the domain of life to which a 
learning activity applies (worklife or personal 
life, current or future job, etc.), and the in- 
tended or expected benefits that motivate par- 
ticipation; we also need to ask about these 
issues separately or combine them in a more 
coherent manner. 

• We need more information on the wide range 
of external inducements (e.g., requirements) 
that motivate participation, both in general and 
for specific coursetaking activities. 

• We need valid and reliable data on the factors 
that inhibit participation, particularly to distin- 
guish those who face external (situational and 
institutional) barriers from those who face in- 
ternal (motivational or dispositional) barriers. 

How to Fill Data Gaps 

• Develop a model for the learning decision- 
making process that includes the role of inter- 
nal goals/incentives, external goals/incentives, 
disincentives or inhibiting factors, and the 
range of learning options to which an adult 
has access. In-depth interviews with a small 
sample of adults, using an ethnographic ap- 
proach, would be appropriate for this purpose. 
Use this model to guide the following three 
activities. 

• Experiment with different ways of asking about 
domains of learning and intended benefits, 
and add appropriate, consistently defined 
items to NHES/AE and other relevant surveys 
(most notably NAAL and ILSS). 

• Potential survey items may exist in other 
countries’ surveys (Finland, Germany, 
England). 

• Experiment with new items on inducements 
other than intended benefits and add appro- 
priate items to NHES/AE and other relevant 



surveys. (NHES staff are currently conduct- 
ing a study on this issue.) 

• Potential survey items may exist in NELS 
and other countries’ surveys (Canada, Fin- 
land, Germany, United Kingdom). 

• Experiment with new barriers items and add 
appropriate items to NHES/AE and other rel- 
evant surveys. 

• Possible survey items may exist in other 
countries’ surveys (Canada, Finland, Swit- 
zerland, United Kingdom). 

6. Investments in Adult Learning 

A basic issue underlying all stages of formal learn- 
ing — adult learning as well as compulsory and 
postsecondary education — is the extent to which 
financial concerns limit access or create inequi- 
ties in the distribution of learning opportunities. 
To address this issue in adult learning requires 
information on learner costs and provider sup- 
port. 

6a. Who pays the financial costs for adults' par- 
ticipation in learning activities (particularly work- 
related activities) — what are the financial costs 
to individuals, employers, labor unions, and fed- 
eral or other government agencies? 

6a1 . How much do participants pay to participate 
in adult learning activities, particularly in adult 
basic education programs, and what do these 
participant costs cover (transportation, childcare, 
books or supplies, etc.)? 

6a2. How often and to what extent do employers 
invest in workforce training, and what are the 
trends over time in employers’ investment in 
workforce training? 

6b. How do adult learners finance their educa- 
tion activities? 

6c. How much and what types of employer sup- 
port and incentives are offered for adult learning 
activities (e.g., granting time off, linking bonuses 
or raises to education or training, partnerships 
with education institutions, training stipends, tu- 
ition reimbursements)? 

Data Currently Available 

In some cases, it is easy for adults to identify who 
covers the costs of their participation in learning 
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activities; they can tell us their own costs, and 
they typically know whether their employer is sub- 
sidizing instruction or whether they receive post- 
secondary student aid. Instruction subsidized by 
federal or state training programs may be identi- 
fiable by the learner as well, since these programs 
sometimes involve enrollment criteria and forms 
that make their government sponsorship evident. 
In other cases, however, the participant will not 
know who pays instructional costs; this is particu- 
larly likely in the case of collaborative arrange- 
ments, e.g., between an employer and an equip- 
ment supplier, an employer and postsecondary 
institution, or a postsecondary institution and 
government or philanthropic agency. Thus, sur- 
veys of adults provide limited information on fund- 
ing sources. Ideally, such information should be 
supplemented by data from employers and post- 
secondary institutions in order to obtain a more 
accurate picture of how financial costs for adult 
learning are covered. 

That said, information on who covers costs is fairly 
well addressed in existing surveys of adults. Data 
on postsecondary attendance costs typically in- 
clude questions about whether the adult received 
any loans, grants, or scholarships. The NPSAS 
(and its follow-ups) are particularly thorough in 
this regard. These surveys include both partici- 
pant and provider data, including student finan- 
cial aid applications and records, institutional aid, 
tuition costs, and participant data on the receipt 
of employer or other aid and expenses paid in 
addition to tuition. 

For other types of adult learning, the NHES/AE, 
IALS, ILSS, SIPP, and NLS-Y all include ques- 
tions about attendance costs. The NHES/AE asks 
how much the adult paid to attend, and whether 
the employer covered some or all costs. The SIPP 
also asks about out-of-pocket costs and employer 
support. The IALS asks whether any of the fol- 
lowing groups covered course fees: adult or fam- 
ily, employer, government, union or professional 
organization, other, or “no fees”; the ILSS will ask 
a similar question. The NLS-Y and SIPP include 
options for specific federal funding programs 
(JTPA, Veterans Administration, etc.). 

Although a few surveys (NHES/AE, SIPP, NLS- 
Y) ask participants how much they pay to attend 
courses, detailed information on what these 



participant costs cover (tuition and fees, supplies, 
transportation, childcare, etc.) is collected only 
on the SIPP and (for postsecondary students) 
NPSAS and its follow-ups. 

The earnings employers forego during the time 
their employees spend in learning activities may 
be the biggest investment that employers make 
in adult learning. Thus, it is important to know 
whether a course is taken during work hours; most 
surveys do ask this. Additional data on the extent 
to which employers invest in workforce training 
are available from the two employer surveys (NES 
and SEPT). Both surveys collect information on 
the amount of money employers invest in train- 
ing. The NES asks for a global estimate, and may 
be of questionable reliability. The SEPT breaks 
out types of costs explicitly and asks for factual 
amounts, so it is probably more reliable. 

The NES and SEPT also provide the best data 
on employer support and incentives. The SEPT 
asks about a long list of training offerings and 
incentives, including tuition reimbursement, train- 
ing resource center, training plans, mentoring 
programs, and contributions to union or associa- 
tion training funds. The 1994 NES focused on 
training plans and offerings; later versions asked, 
for each of five groups of workers (managers, 
supervisors, etc.) whether the employer offers (or 
whether employees received) tuition reimburse- 
ments or time off for taking classes. The collec- 
tion of these NES data by employee group is an 
improvement over more global questions, since 
not all workers may have equal access to such 
benefits. Since neither of these surveys is sched- 
uled to be repeated (beyond 2000), trend data 
(or anyfuture data) on these issues are not avail- 
able. 

As mentioned above, some surveys ask adults 
about employer support for the courses they take; 
this can provide an indirect indicator of overall 
employer support. The NHES/AE asks partici- 
pants whether they received any of the following 
types of employer support: requirement to take 
course, time off with or without pay, classroom 
space, or payment of all or some costs. The NELS 
asks participants whether the employer provided 
tuition aid or financial assistance for attending an 
education institution. Finally, as mentioned above, 
the NPSAS includes information about support 
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received from employers for postsecondary 
coursetaking. 

Most Important Data Gaps to Fill 

• We need more specific break-out of costs in- 
curred by participants. 

• We need more complete information on em- 
ployer support and incentives. 

How to Fill Data Gaps 

• Revise NHES/AE question on costs to include 
costs for tuition and fees, books and supplies, 
and possibly other categories (e.g., transpor- 
tation, childcare). 

• Potential survey items may exist in SIPP and 
other countries’ surveys (United Kingdom). 

• Conduct a survey of employers, preferably in 
collaboration with BLS, to collect information 
on employer support and incentives for learn- 
ing. This survey could also address a number 
of other important issues, such as employer 
offerings, instructional delivery methods, col- 
laborative relations among providers, em- 
ployer-specific skills training, and reliance on 
formal versus informal learning methods. 

7. Adult Learning Providers 

Adult learning can occur in many settings, with 
many sponsors; thus, the learning provider is a 
basic descriptive feature of adult learning. Of par- 
ticular interest is the extent to which employers 
and postsecondary institutions serve as instruc- 
tional providers, for all adults or for particular 
groups of adults. 

High Priority Issues 

7a. What organizations (1) offer learning activi- 
ties for adults, (2) fund learning activities for 
adults, and (3) provide curricula for adult learn- 
ing activities? In other words, who is the site pro- 
vider, the funding provider, and the curriculum 
provider — particularly for adult basic education 
and work-related education? Who is moving into 
and out of each of these markets? 

7b. How extensive a role do formal education in- 
stitutions and employers play in the provision of 
adult learning activities? How are these roles 
changing over time? 



7b1 . What is the role of postsecondary institu- 
tions, overall and by institutional type, in the larger 
service delivery system for adults? What propor- 
tion of adult learners do they serve, and which 
learners do they serve? What is their role in the 
provision of adult basic education, work-related 
education, and avocational education for adults? 

7b2. Are employers offering more or less training 
to employees now compared to in the past? If 
less, are employees making up for that training 
loss elsewhere? 

7c. Where do workers get their education and 
training?\Nha\ proportion of worker education is 
provided by employers, labor unions, or higher 
education? What proportion is provided by col- 
laborations of these groups? 

7d. For each type of adult basic education pro- 
gram (ABE, ASE, ESL), what proportion of the 
teaching force is voluntary versus paid staff, and 
full-time versus part-time staff? 

7e. What proportion of the adult basic education 
teaching force has formal education training? 
What proportion has formal training to educate 
adults? 

7e1 . What proportion of the adult basic educa- 
tion teaching force has formal training in diag- 
nosing and instructing students with learning dis- 
abilities? How well-prepared are adult education 
instructors to adapt teaching for students with 
learning disabilities? 

7e2. What proportion of the adult basic educa- 
tion teaching force has formal training in alterna- 
tive learning styles and instructional methods? 
How well-prepared are adult education instruc- 
tors to adapt their teaching to adults’ learning 
styles? 

Data Currently Available 

Most surveys of adult learning collect informa- 
tion on course providers. The NHES/AE includes 
a relatively long list of providers for each reported 
learning activity. For ESL and basic skills instruc- 
tion, the NHES/AE collects information on both 
the provider of instruction and the location where 
the course was taken; for other courses, only the 
instructional provider is asked about. The ILSS, 
SIPP, NLS-Y, and BPS/B&B also include lists of 
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providers, although their lists are less extensive 
and are not always linked to specific courses. 
These surveys typically ask only for the instruc- 
tional provider. The list in NLS-Y combines learn- 
ing site with instructional provider (e.g., training 
seminar offered outside of work). For most pur- 
poses, it may be necessary only to distinguish 
between a limited number of providers, such as 
postsecondary institutions, employers, commer- 
cial agencies, government agencies, and labor 
unions. Thus, the more abbreviated lists in ILSS, 
SIPP, NLS-Y, and BPS/B&B may be sufficient for 
most purposes; from these data, we can deter- 
mine the role that postsecondary institutions and 
employers play in the provision of adult learning 
opportunities. 

One problem with all provider data is that the re- 
spondent may not always be able to distinguish 
who the “provider" is — particularly who sponsors 
(pays for) the instruction versus who provides 
(teaches) it. For example, an employer might pay 
for a community college to offer a customized 
training course at the worksite; depending on how 
the survey questions are worded and how well 
the participant understands the instructional ar- 
rangements, the participant could list either the 
community college or the employer as the pro- 
vider. (First, of course, the analysts designing the 
survey must know which provider they are inter- 
ested in knowing about; it is not clear that this is 
always accomplished.) Another problem is that 
not every survey asks for the same provider (e.g., 
the NHES/AE asks for instructional sponsor, SIPP 
for funding sponsor). 

In sum, we can get information that should be 
reasonably valid in most cases for who provides 
learning opportunities from the NHES/AE and 
ILSS, and for a limited population of adults from 
BPS/B&B. In addition, the SIPP and NLS-Y pro- 
vide reasonably good data on providers of work- 
related training. But these data are limited by their 
dependence on respondents’ knowledge of pro- 
vider arrangements. And as these arrangements 
become more complex (e.g., through contract 
training), current provider questions will become 
less satisfactory. The most obvious alternative is 
to rely on surveys of providers, particularly for 
information on funding and curriculum sources. 
Barring a universe list of all providers, surveys of 



postsecondary institutions and employers are 
best for collecting this information. Such provider 
surveys offer the best means to monitor collabo- 
rative arrangements and the roles of postsecond- 
ary institutions and employers in providing fund- 
ing, curriculum, and other instructional support. 
Better yet would be surveys of providers linked 
with surveys of adult participants (as was done 
in the SEPT and is done in NPSAS); this strategy 
provides an especially rich data source for mod- 
eling the learning decisionmaking process and 
understanding access issues. 

Any survey of adults that lists providers (includ- 
ing postsecondary institutions and employers) 
could be used to compare the incidence of for- 
mal education and employer-sponsored training. 
The NHES/AE, ILSS, SIPP, and NLS-Y (the lat- 
ter two for work-related training only) are prob- 
ably the best surveys for this purpose. To com- 
pare the magnitude (intensity) of formal educa- 
tion versus employer-sponsored training would 
require that comparable instructional-time mea- 
sures be collected for each learning activity. The 
NHES/AE has the best data in this regard, but 
there are reliability and validity problems with in- 
structional-time data (see issue area 4). 

Questions about offerings can only be addressed 
through surveys of employers. The BLS has con- 
ducted a few employer surveys over the years, 
but no future employer surveys are planned. The 
NES has data on offerings for 1 994 and 1 997 (and 
possibly in 2000), but no future surveys are 
planned beyond 2000. We can approximate 
changes over time in employer offerings if we 
assume that changes in participation in employer- 
provided training reflect changes in offerings 
(rather than in employees’ propensity to partici- 
pate in offerings). For this purpose, the SIPP and 
NHES/AE provide trend data on participation in 
employer-sponsored training. Another approach 
is to examine changes over time in the propor- 
tion of adults who participate in learning activi- 
ties that receive employer support. For this ap- 
proach, the data sets that contain information on 
employer support (NHES/AE, NELS) could be 
used. 

Evaluating where workers get their education and 
training can be more difficult than it seems. The 



Jfelong Learning 



typical survey of adults (such as NHES/AE and 
ILSS) asks about education and training experi- 
ences over an extended period of time (e.g., 12 
months), but ask only about current work status — 
so we cannot always tell if someone was em- 
ployed by their current employer at the time of 
their participation, or where they were employed 
at that time. In most cases, this is probably not a 
problem. But more accurate and informative data 
could be obtained by asking about training activi- 
ties within the context of the adult’s work history 
(as in NLS-Y), or by asking about training that 
was received to improve skills “while working,” as 
is done in SIPP (other questions are asked about 
training to obtain a new job). 

There are no national data on the adult basic 
education teaching force, other than what may 
be included in past national evaluations of adult 
literacy programs. It is difficult to collect informa- 
tion on these instructors because of the variety 
of organizations and agencies for which they work. 
Data are available on postsecondary instructors’ 
formal training in adult education (in IPEDS and 
NSOPF), but these data cannot be linked to 
whether an instructor teaches adult basic educa- 
tion. Such information could be added to the 
NSOPF, and parallel information could be added 
to the Schools and Staffing Surveys to find out 
which K-1 2 teachers provide adult basic educa- 
tion. However, even with more information on 
K-1 2 and postsecondary teachers, our data 
would be incomplete, as adult basic education 
courses may be taught by retired teachers or other 
adults with no formal teaching experience. Two 
other possible data sources exist. First, state of- 
fices of adult education could be surveyed about 
their requirements for state adult basic educa- 
tion (ABE) grantees, including the educational re- 
quirements for instructors. Second, national evalu- 
ations of ABE programs could be reviewed and 
national adult literacy organizations could be que- 
ried to learn more about staff qualifications (and 
possible procedures for creating a universe list 
of ABE providers). 

Most Important Data Gaps to Fill 

• Questions targeted to adults about providers 
need to be standardized to yield the most valid 
and reliable data possible, including options 
for respondents to unambiguously indicate col- 
laborative arrangements among providers. 



How to Fill Data Gaps 

• Experiment with new ways of asking questions 
about providers, ideally in combination with 
questions about employer support; revise 
NHES/AE, ILSS, and NAAL (and other rel- 
evant survey) questions accordingly. 

8. Instructional Delivery and 
New Technologies 

New technologies have enormous potential for 
increasing access to both formal and informal 
learning opportunities, and for minimizing ineq- 
uities in learning opportunities among certain 
groups of adults, such as those in rural locales or 
with physical disabilities. This potential makes the 
use of technologies in instruction an important 
characteristic to monitor. 

High Priority Issues 

8a. What instructional delivery methods are used 
in adult learning activities?To what extent are new 
technologies (including the Internet, CD-ROMs, 
other computer-based systems, and telecommu- 
nications) used as a delivery mechanism for adult 
learning? 

8b. How are technology-based instructional pro- 
grams developed and offered? How many orga- 
nizations that offer technology-based instruction 
for adults provide both the curriculum and instruc- 
tional delivery system? How many use outside 
vendors for curriculum development? For instruc- 
tors or instructor training? For delivery system 
implementation? 

8b1 . What are the trends in the use of computer 
and telecommunication technologies as a means 
of delivering instruction in adult learning? 

8c. What proportion of adults use new technolo- 
gies at home specifically to acquire new knowl- 
edge or skills? Which technologies do they use, 
and how frequently do they use them? 

8d. How many adults, and which adults, have 
participated in an adult learning activity utilizing 
a technology-based delivery system? 

8d1 . How many adults are enrolled in adult basic 
education programs that use computer or tele- 
communications technologies? 

8e. What proportion of employers offer their em- 
ployees technology-based instructional activities? 
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What types of technology-based delivery systems 
are used? 

8f. What proportion of postsecondary institutions 
offer technology-based courses for adults ? What 
types of technology-based delivery systems are 
used? To which groups of adults (e.g., workers 
seeking new skills, professionals in continuing 
education, adults in remote areas, ESL adults) 
are these courses targeted? 

8f1 . What are the characteristics of postsecond- 
ary institutions that offer technology-based in- 
structional programs (e.g., location, type of insti- 
tution)? 

8g. How many postsecondary institutions (and 
corporate universities) offer distance education 
programs, and how many adults participate in 
these programs 7 To what remote sites is distance 
education delivered (e.g., students’ homes, 
branch worksites or campuses, elementary or 
secondary schools)? To what populations is it 
targeted? 

8h. What proportion of potential providers (such 
as schools, postsecondary institutions, libraries, 
and community-based organizations) have ad- 
vanced technologies that are capable of instruc- 
tional delivery? 

Data Currently Available 

Two surveys (one ongoing) have detailed infor- 
mation on instructional delivery methods, one for 
employer-provided training (SEPT) and one for 
postsecondary education (NSOPF). The SEPT 
asked employees whether their training was pro- 
vided through classroom instruction, small-group 
or one-on-one discussion, a computer tutorial, 
video instruction, hands-on learning, or observa- 
tion. The last administration of the SEPT was in 
1995; given the rapid pace of change in instruc- 
tional technologies, SEPT findings are probably 
no longer relevant for describing current em- 
ployer-sponsored training. For (for-credit) postsec- 
ondary education courses, the NSOPF distin- 
guishes among classes taught by lecture or dis- 
cussion, seminar, lab or clinic, apprenticeship or 
field work, or “other”; this survey also distinguishes 
among courses that involve face-to-face, com- 
puter, TV-based, or other interactions. If all post- 
secondary education is considered adult learn- 
ing, the NSOPF is a useful source of information 



on instructional delivery. However, if one wants 
to exclude the traditional or full-time student, 
NSOPF is of less use, since these students are 
not identified within this survey. (NSOPF data can 
be linked to faculty who teach more or fewer con- 
tinuing education courses.) 

Limited data are available to address the issue of 
how technology-based programs are developed 
and offered. To thoroughly answer this question 
requires a survey of providers. One provider sur- 
vey does provide relevant information. The PEQIS 
surveys on distance education (conducted in 1995 
and 1998) collect information on the prevalence 
and nature of this form of technology-based in- 
struction. Additional data are available from sur- 
veys of adults, which provide information on the 
extent to which adults participate in technology- 
based programs. As mentioned above, the em- 
ployee component of the SEPT asks if training 
was received via computer or video instruction, 
but these data are not available beyond 1 995. The 
1999 NHES/AE asks if instruction (in every area 
except apprenticeships) occurred through dis- 
tance education, and if so, the specific technolo- 
gies used for student-instructor communications. 
The ILSS may ask if courses were taken through 
distance education, and if so, with what method. 
However, the options for specifying the mode of 
distance education in the NHES/AE and ILSS 
differ greatly. The former includes radio or TV, 
e-mail, Internet, satellite broadcasts, video 
conferencing, or computer conferencing. The lat- 
ter includes television, radio or videotapes, 
Internet, or correspondence. 

The best data on postsecondary institutions’ tech- 
nology-based offerings come from the PEQIS sur- 
veys. On the 1995 survey, courses for adults can 
be fairly well approximated. One question identi- 
fies the number of distance education courses 
targeted to adult basic education students, pro- 
fessional continuing education students, and 
other continuing education students; another 
question asks if professional or other workers are 
targeted by any distance education courses. (It 
would be helpful to add “workers” to the first ques- 
tion on number of courses.) The type of institu- 
tions that offer distance education can be identi- 
fied through the PEQIS link to IPEDS data. The 
1998 version of the PEQIS distance education 
survey does not provide information on students 
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served by these programs, and so cannot be used 
to determine distance education offerings for spe- 
cific groups of adults. (This change reflects a 
change in priorities on the survey, rather than 
problems with the original items.) 

In short, there is no source of information on the 
full range of instructional delivery methods used 
in adult learning activities; however, data do exist 
on distance education within the NHES/AE and 
PEQIS surveys. There is no information on em- 
ployer involvement with distance education (other 
than indirectly through questions in the 1995 
PEQIS). But since this is probably a very limited 
delivery method among employers, this lack of 
data may not be a problem. 

Most Important Data Gaps to Fill 

• In general, we need better data on all nontra- 
ditional forms of adult learning, including learn- 
ing delivered via new technologies. Given the 
rapid rate of change in these technologies, this 
information should be collected relatively fre- 
quently. 

• We need to assess the extent to which dis- 
tance education is targeted to adult learners, 
since this instructional delivery system could 
be a particularly effective means of overcom- 
ing the time and location constraints faced by 
these individuals. 

How to Fill Data Gaps 

• Develop and add questions to surveys of 
adults (e.g., NHES/AE, NAAL) to monitor the 
use of new technologies in instruction, includ- 
ing but not limited to distance education (e.g., 
use of the Internet). 

• Regularly administer the Distance Education 
PEQIS; review and revise as necessary this 
survey’s ability to identify adult learners and 
workplace skill development. 

9. Informal Learning 

Formal learning is assessed more often than in- 
formal learning, probably because it is more 
amenable to policy intercession and because it 
is easier to measure. But informal learning has 
always been an important form of adult learning. 
Informal learning may become even more impor- 
tant as learning becomes increasingly special- 



ized and in need of constant updating, and as in- 
formation sources (such as CD-ROM programs 
and the Internet) become more widely accessible. 

High Priority Issues 

9a. What informal education and training meth- 
ods do workers use? 

9b. How much of adult learning is informal ver- 
sus formal? In what situations and for what types 
of knowledge/skill development is informal learn- 
ing engaged in rather than formal learning? 

9b1 . To what extent do workers use formal or in- 
formal methods to develop and update their skills 
and knowledge? 

9b2. What shifts are occurring over time in adults ’ 
participation in formal versus informal learning? 
Are there greater shifts among some types of 
adult learning (e.g., basic skills) or for some 
groups rather than others (e.g., by SES or occu- 
pation)? 

9b3. What is the relationship between participa- 
tion in formal and informal education? That is, are 
formal and informal education complementary or 
mutually reinforcing — do those who engage in 
more of one type engage in more or less of the 
other type? 

Data Currently Available 

Most surveys collect information only on formal 
learning activities. Currently, the only exceptions 
to this are the SEPT and NLS-Y, both of which 
ask about formal and informal work-related ac- 
tivities. (The other survey that is focused on work, 
SIPP, does not distinguish between formal and 
informal activities; it asks about “any training to 
improve job skills” and “any training to get a new 
job”) Both the SEPT and NLS-Y appear to be 
good sources for comparing the relationships 
between formal and informal training. 

The ILSS proposes to include a few questions on 
participation in informal work-related training, so 
in the future data may be available from NLS-Y 
and ILSS. Data from the SEPT are the most in- 
formative, however. For example, the SEPT asked 
workers how they learned the skills required on 
their current job— through formal training, infor- 
mal training, or on their own — and which method 
was most important. No similar data are avail- 
able in ongoing surveys. 
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Reliability is a major issue for data on informal 
learning. Because these activities tend to be less 
structured, of relatively short duration, and often 
embedded into one’s normal work or life activi- 
ties, it is easy for learners to forget that they par- 
ticipated in such activities. The SEPT avoided this 
problem by collecting participation data using em- 
ployee logs (completed daily over 10 days); the 
NLS-Y minimizes the problem by asking about 
informal activities taken only within the past 4 
weeks. (There appears to be less concern with a 
“seasonality” bias for informal rather than formal 
activities.) The ILSS proposes to ask about infor- 
mal activities within the past 1 2 months, so these 
data may not be as reliable as the NLS-Y or SEPT 
data. 

Not surprisingly, current definitions and catego- 
ries for informal work-related learning vary by 
survey. The SEPT counted as informal activities 
those that were unplanned and unstructured, 
excluding self-instruction. The NLS-Y defines in- 
formal training as that received from a supervi- 
sor or coworker, or self-paced instruction. The 
ILSS plans to include conferences, seminars, 
shop/quality circles, supervisor or coworker in- 
struction, and self-learning. This is obviously an 
(other) area of lifelong learning that is as yet not 
clearly defined. Because of the lack of consen- 
sus on how to define informal learning, we do not 
propose that NCES attempt to develop items on 
informal avocational learning at this time. None- 
theless, to fully describe all forms of adult learn- 
ing, we will need to eventually capture this part 
of adults’ learning experiences. 

Most Important Data Gaps to Fill 

• We need better data on work-related informal 
learning, collected in a consistent manner. 

• We need to collect data on informal work-re- 
lated learning more frequently than is currently 
done (using the NLS-Y and ILSS). 

How to Fill Data Gaps 

• Develop a standard format with which to ask 
about informal work-related education, using 
existing questions and data as a guide. 

• Potential survey items may exist in SEPT, 
NLS-Y, ILSS, and other countries’ surveys 
(Finland, Germany, Switzerland, United 
Kingdom). 



• Add questions on informal work-related learn- 
ing to more frequently administered surveys, 
such as the NHES/AE and/or NAAL. 

10. Services and 
Accommodations for Adults 

Given their work and family responsibilities, adult 
learners often have special needs that are differ- 
ent from or more extensive than those of younger 
students. To ensure reasonable access to adult 
learning activities, providers need to offer pro- 
grams and services that address these needs. 
To monitor access, we need to track these ser- 
vices and accommodations. 

High Priority Issues 

1 0a. To what extent are postsecondary institutions 
providing alternative offerings or delivery meth- 
ods that meet the needs of adults? To what ex- 
tent are they working collaboratively with employ- 
ers or vendors to provide contract or customized 
training, “just-in-time" training, and/or worksite- 
based training? 

1 0b. To what extent are postsecondary institutions 
and other providers offering competency-based 
assessment, portable credentials, or other out- 
come-based approaches that allow adults to 
document their skills and knowledge acquisition? 

10c. To what extent are postsecondary institutions 
and other providers offering support services or 
other special accommodations to improve access 
for adult learners, or to otherwise better serve 
adult learners (e.g., special counseling services 
for adults, transportation services, childcare, ex- 
tended hours for student service offices, stream- 
lined registration procedures, evening and week- 
end offerings)? 

1 0d . What is the extent of noncredit coursetaking 
within postsecondary institutions? Which institu- 
tions offer it and for what purposes (to generate 
revenues, for public service, to support economic 
development, etc.)? 

1 0dl . How many adults, and which adults, par- 
ticipate in noncredit coursetaking at postsecond- 
ary institutions, and for what purposes? What 
trends are occurring in noncredit coursetaking? 

Data Currently Available 

Other than the data discussed above on distance 
education, there are no data on alternative offer- 



.ifelong Learning 



ings and delivery methods used by postsecond- 
ary institutions to serve adults. To address this 
issue, we would first have to specify the alterna- 
tive offerings and delivery methods that are of 
interest, then add the relevant questions to exist- 
ing postsecondary surveys or to one or more 
PEQIS surveys. Since most postsecondary insti- 
tutions do attempt to meet adults’ needs, the fo- 
cus should be on those offerings that are of great- 
est policy relevance and on discerning in what 
ways and how much of an institution’s program- 
ming is adapted to meet adults’ needs. Although 
a survey of providers would be the best source of 
data on postsecondary services, offerings, and 
credentialing methods, a survey of adults could 
provide related information on the extent to which 
these offerings are used or desired, or to which 
their absence is perceived to be problematic. 

Few data exist on noncredit coursetaking. To es- 
timate the extent of noncredit coursetaking within 
postsecondary education, the NSOPF provides 
counts of how many remedial and continuing edu- 
cation courses faculty teach that are noncredit. 
This will probably yield an undercount of noncredit 
courses, however, as these courses are often 
taught by teaching assistants, a group that is not 
included in the NSOPF. A recent effort to develop 
a PEQIS survey on noncredit coursetaking was 
abandoned after it was discovered that in many 
institutions, no one person could report noncredit 
offerings across the institution; these courses are 
often kept “off the books” or reported separately 
by department. 

In theory, the NHES/AE could be used to esti- 
mate how many and which adults participate in 
noncredit coursetaking, but the current structure 
of the survey does not lend itself to this type of 
analysis. Although the NHES/AE separates out 
courses taken as part of a postsecondary cre- 
dential program, all other courses taken at post- 
secondary institutions are not distinguished as 
for-credit or not-for-credit. 

Most Important Data Gaps to Fill 

• We need to collect data on services and ac- 
commodations that are offered for adult learn- 
ers, particularly those services and accommo- 
dations that are likely to improve access for 
traditionally under-represented groups. (Note 
that this data need is related to the need for 
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better data on disincentives and barriers, as 
discussed in issue area 5 above.) 

• We need better data on nontraditional forms 
of adult learning, including noncredit postsec- 
ondary coursetaking, contract training, and 
alternative assessment and credentialing pro- 
cedures. 

How to Fill Data Gaps 

• Develop and regularly administer a PEQIS 
survey on postsecondary services and accom- 
modations for adults. 

• Expand/modify questions on postsecondary 
student and provider surveys (IPEDS, NPSAS/ 
BPS/B&B, NSOPF) to collect better data on 
nontraditional forms of adult instruction, as- 
sessment, and credentialing. 

• Restructure NHES/AE so that it can provide 
counts of noncredit coursetaking at postsec- 
ondary institutions. 

• Examine the feasibility of a PEQIS on con- 
tract training to examine this form of instruc- 
tional delivery. (Note that this survey could also 
address more general questions about col- 
laborations among providers.) 

• Consider a PEQIS variation that would 
ask department chairs about noncredit 
coursetaking, and/or other nontraditional as- 
sessment or credentialing procedures. 

• Potential survey items on some of these top- 
ics may exist in other countries’ surveys (Ger- 
many, United Kingdom). 

11. Outcomes and Effectiveness 

Learning is intended to produce benefits for the 
individual, benefits that can range from something 
as inconsequential as an enjoyable way to pass 
one’s time to something as substantial as in- 
creased earnings. Employers also expect finan- 
cial benefits from the provision of worker train- 
ing. Monitoring learning outcomes documents 
the value of adult learning activities and helps 
explain why adults choose to participate in this 
activity. 

High Priority Issues 

11a. To what extent do adults use the skills and 
knowledge gained from an adult learning activity, 
either at work or in their personal life? 



Lifelong Learning 

32 



11b .To what extent do adults judge their learning 
activities to provide specific job-related benefits 
or returns, such as increases in earnings ability, 
better job satisfaction, improved ability to func- 
tion at work, greater opportunity for job mobility? 

11c .To what extent do adults participate in learn- 
ing activities that result in a credential, and what 
type of credential do they obtain (e.g., degree, 
state license, institutional certificate, company 
certificate)? 

1 1 d. What are the economic returns to adults for 
participation in work-related learning activities? 
What economic costs and benefits do adults ac- 
crue from their participation? 

1 1dl . What are the economic returns to educa- 
tion for adults at different education attainment 
levels? What are the trends in these returns over 
time? 

1 1 d2. What are the economic returns to employ- 
ers for investing in employee training and educa- 
tion? 

1 1d3. Does the availability of employer-provided 
training make an employee more or less likely to 
remain with an employer? To what extent does 
less opportunity for training encourage employee 
turnover, and to what extent does high turnover 
discourage the provision of formal training? 

Data Currently Available 

Adults’ reports of the extent to which they use 
skills acquired in learning activities are collected 
in IALS, probably in ILSS, and in NLS-Y and SIPP. 
(The SIPP asks which types of learning activities 
were most useful on the job.) If we rely on IALS/ 
ILSS for these data, they will be available on a 
1 0-year cycle. 

In terms of specific job outcomes, the NELS asks 
participants about a range of benefits received 
from participation (such as increased opportuni- 
ties for promotion, wider range of job options, in- 
creased pay, and increased job responsibilities 
or effectiveness). The ILSS may include a similar 
question. It would be advantageous to also have 
such a question on NHES/AE or NLS-Y, where a 
broader range of adults and/or employment data 
is available on a more routine basis. 

Complete data on the credentials adults either 
seek or receive through adult learning activities 



are not available. Many surveys ask about the 
education credentials that were sought and/or 
received. The only surveys that ask about other 
credentials received (professional licenses, cer- 
tificates, etc.) are the BPS/B&B, which are re- 
stricted to adults who have enrolled in college (or 
received a bachelor’s degree). The NELS provides 
some general information on this issue, since it 
asks about the attainment of various types of pro- 
fessional licenses since leaving high school, but 
these attainments cannot be linked to specific 
coursetaking. It is also not clear that company 
certificates, such as a Novell Technician Certifi- 
cate, would be reported on NELS. The ILSS may 
include a question about credentials, but in a form 
that is too broad to be very useful. 

While it is not feasible for NCES to conduct the 
type of controlled study that could determine the 
economic returns to adult participation in work- 
related learning activities, there are data sources 
that can provide strong evidence on this topic. 
The SIPP and NLS-Y both contain longitudinal 
data that allow one to track participation in work- 
related learning, along with changes in income 
level and employment status. The NHES/AE, 
NAAL, and ILSS could also provide suggestive 
evidence, particularly if questions were added to 
these surveys about income level prior to the 1 2- 
month reference period. (Changes in income 
could be compared for those who did and did not 
participate in learning activities over the 12-month 
period, controlling for other factors that influence 
participation, such as employer size.) 

The NLS-Y is the best data source for examining 
the effects of training on employee turnover. This 
survey asks about training activities within the 
context of the adult’s work history, thus linking 
training to particular employers, while also track- 
ing changes in employment over time. 

Returns to education for adults at different edu- 
cation attainment levels have been fairly well- 
studied, most notably in the NCES report Educa- 
tion and the Economy: An Indicators Report. To 
further examine this issue, any data set on the 
adult population can be used as long as it includes 
individual-level measures of education, income/ 
salary, and age and/or work experience (the lat- 
ter two as controls). Data sets meeting this crite- 
rion include the CPS, ACS, NHES/AE, NAAL, 
ILSS, SIPP, NLS-Y and NELS. However, recent 
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evidence suggests that returns to education may 
be mediated by the nature of one’s job — specifi- 
cally, by whether or not the job is technical in na- 
ture. Teasing out this type of mediating effect 
would require detailed information on occupations 
(suggesting the need to link to 0*NET data). 

Most Important Data Gaps to Fill 

• We need to make use of available information 
on labor market returns. 

• We need better data on the specific outcomes 
participants achieve (based on their own per- 
ceptions). 

• To examine changes in income among adults 
who do and do not participate in learning, we 
need data on income prior to the survey refer- 
ence period. 

How to Fill Data Gaps 

• Analyze SIPP and NLS-Y data to provide in- 
formation on the labor market returns to par- 
ticipation in (work-related) adult learning ac- 
tivities. 

• Add items to NHES/AE to determine the ex- 
tent to which adults participate in learning to 
receive alternative credentials (such as those 
awarded by a company or industry). 

• Potential survey items may exist in other 
countries’ surveys (Germany, United King- 
dom). 

• Experiment with questions asking about spe- 
cific job benefits, using NELS as a guide, then 
add appropriate questions to NHES/AE, 
NAAL, and/or ILSS. 

• Potential survey items may exist in other 
countries’ surveys (Canada, Finland, Ger- 
many, Switzerland, United Kingdom). 

• On the NHES/AE, ILSS, and other surveys that 
collect participation data over a given refer- 
ence period, add a question about income 
level prior to the reference period. 

• Consider linking data on intended benefits 
(see issue area 5 above) to data on actual 
benefits, as a measure of effectiveness. 



market forces balance the supply of adult learn- 
ing opportunities with the demand for such op- 
portunities. However, as the need for adult learn- 
ing rises, this assumption is increasingly ques- 
tioned, and the government has taken a more 
active role in fostering adult learning. Policymak- 
ers have an obvious interest in knowing the ex- 
tent to which their efforts are effective and ap- 
propriate. 

High Priority Issues 

1 2a. To what extent do adults use the Hope Schol- 
arship and the Lifelong Learning tax credit? Have 
these recent federal initiatives increased partici- 
pation in adult learning? 

1 2b. How does government and corporate sup- 
port for adult training and education (particularly 
for workers) in the U.S. compare to that in other 
countries? 

Data Currently Available 

The 1999 NHES/AE has data on the use of the 
Hope Scholarship and the Lifelong Learning Tax 
Credit. NPSAS 2000 will also collect information 
on the use of these incentives among college stu- 
dents. We do not know if the SIPP or NLS-Y sur- 
veys will include questions about these initiatives 
in future administrations, but they would be use- 
ful surveys in which to add such items, because 
of their large samples and longitudinal design. 

The ILSS is attempting to address the issue of 
government and corporate support for adult learn- 
ing across countries, but it is difficult to do so 
through this type of survey, since adults are often 
not aware of the existence or extent of govern- 
ment and corporate support. More complex stud- 
ies undertaken by OECD (e.g., past work of Hong 
Tan) may be more appropriate for addressing this 
issue. 

Most Important Data Gaps to Fill 

• NCES surveys ask about the use of Hope 
Scholarships and Lifelong Learning tax cred- 
its, but more reliable data on these issues 
could be obtained directly from the IRS. 

How to Fill Data Gaps 
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12. Government Role in 
Adult Learning 

Government support for adult learning has been 
relatively minimal, under the assumption that 



To provide the most reliable data on the use 
of federal adult learning incentives, NCES 
should work out a cooperative agreement with 
the IRS to allow NCES access to relevant in- 
formation on the use of such incentives. 
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SECTION 3: RECOMMENDATIONS 



The Lifelong Learning Task Force was charged 
with developing a set of recommendations to help 
NCES management in its efforts to develop a stra- 
tegic plan for the Center’s future work. Given the 
inevitable uncertainties concerning fiscal and 
personnel resources, the task force was asked 
to propose a series of recommendations ranging 
from less to more ambitious. To meet this goal, 
the task force developed a series of recommen- 
dations designed to answer the following ques- 
tions: 

• Should NCES develop a systematic data col- 
lection and reporting system on lifelong learn- 
ing? If so, 

• What data collection instruments should be 
included in this system? 

• What actions should be pursued to develop 
this system, and which of these actions should 
be pursued first? 

• What organizational structures should be used 

to support these efforts? — 

As discussed below, the task force recommends 
that NCES pursue the development of a data 
collection and reporting system on lifelong learn- 
ing, using currently available data collection in- 
struments. The first steps in this effort should be 
to consolidate information available from existing 
data sources while simultaneously improving the 
validity and comprehensiveness of the data pro- 
vided by existing sources. These efforts would 
improve both the lifelong learning data collection 
system and the lifelong learning reporting sys- 
tem. To do this work, we propose that an inter- 
program team be designated to allocate a cer- 
tain percentage of their workload to these efforts. 
It is imperative that staff workloads be adjusted 
to accommodate these tasks within a regular work 
schedule, rather than adding these tasks to a full 
schedule of work. 

Value of Developing a 
Lifelong Learning System 

The review of the literature, views of TRP mem- 
bers, and breadth of current initiatives on lifelong 
learning convinced task force members that adult 



learning has become an area of important and 
lasting policy interest. However, this alone does 
not mean that NCES should develop a data col- 
lection and reporting system on this topic. To de- 
termine whether the Center should pursue a life- 
long learning agenda, we reviewed the data col- 
lection mandates for NCES and BLS, then devel- 
oped a list of “pros” and “cons.” 

Data Collection Mandates 

The mandate for NCES is as follows: 

“The purpose of the Center shall be to collect, 
analyze, and disseminate statistics and other data 
related to education in the United States and in 
other nations.” (Section 406(b) of the General 
Education Provisions Act, as amended (20 U.S.C. 
122le-1) January 1995, emphasis added.) 

This mandate can be interpreted narrowly or 
broadly, depending on how one interprets the 
phrase “data related to education.” Narrowly in- 
terpreted, this phrase refers to the formal educa- 
tion system, including K-12 and postsecondary 
education. These are the existing education sys- 
tems in the U.S., and the areas on which the fed- 
eral government has traditionally focused its edu- 
cation policy initiatives. Based on this interpreta- 
tion, one could argue that NCES should continue 
to focus its efforts primarily on this sector of edu- 
cation. 

Broadly interpreted, “education” refers to a wide 
range of learning processes, all of which contrib- 
ute to the quality of life by improving the social 
and economic functioning of individuals and so- 
ciety as a whole. Learning that occurs outside of 
the formal education system also contributes to 
these ends, and thus also deserves federal at- 
tention. The federal government has traditionally 
paid little attention to this area, except to fund 
adult basic skills education and job training pro- 
grams for disadvantaged individuals. Federal poli- 
cymakers have, however, recently expanded their 
focus to include the education of adults in gen- 
eral, through initiatives such as the Lifelong Learn- 
ing and Hope Scholarship tax credits. 

The Lifelong Learning Task Force endorses the 
broader interpretation of NCES’ mission. We feel 
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this interpretation more accurately captures the 
spirit of the law, and better represents NCES’ 
current work (e.g., the Center’s work on early 
childhood education and on the relationship be- 
tween education and economic outcomes). In our 
view, a lifelong learning agenda does fit within 
NCES’ mandate, or at least under one reason- 
able interpretation of that mandate. 

But could lifelong learning perhaps fall under BLS’ 
mandate? In theory, collecting data on work-re- 
lated adult learning could be considered a respon- 
sibility of BLS, as that office is mandated to: 

"acquire and diffuse among the people of the 
United States useful information on subjects con- 
nected with labor, in the most general and com- 
prehensive sense of that word, and especially 
upon its relation to capital, the hours of labor, the 
earnings of laboring men and women, and the 
means of promoting their material, social, intel- 
lectual, and moral prosperity” (29 U.S.C.A. §1). 

In practice, however, BLS is organized into of- 
fices that collect data on economic indicators that 
are only indirectly related to adult learning: Em- 
ployment and Unemployment Statistics, Prices 
and Living Conditions, Compensation and Work- 
ing Conditions, Productivity and Technology, and 
Employment Projections. Current indications are 
that BLS expects to collect all future data on 
worker training and education exclusively through 
the NLS-Y, which collects data only from a co- 
hort of adults. Given this focus, BLS does not 
appear to be a feasible source for systematic and 
complete information on lifelong learning. 

With that background, the Lifelong Learning Task 
Force generated the following “pros” and “cons” 
associated with the development of a lifelong 
learning data collection and reporting system. 

Pros 

• Adult learning appears to be an area of grow- 
ing, permanent importance, making data on 
this topic important to monitor for general 
policy purposes. 

• More specifically, this data collection system 
would provide data relevant to a number of 
recent policy initiatives, such as the Hope 
Scholarship tax credit, the Lifelong Learning 
tax credit, and the Workforce Investment Act 



(WIA, which emphasizes linking adult educa- 
tion and worker training). 

• A lifelong learning data collection system 
would explicitly recognize that education and 
learning occur beyond the formal education 
system, and that these learning activities are 
an important part of the U.S. social and eco- 
nomic systems: in other words, it would help 
foster the goal of creating “a nation of learn- 
ers.” 

• The Center is well-positioned to address this 
topic, and doing so would build on existing 
adult education efforts, including the NHES/ 
AE, national and international adult assess- 
ments, and postsecondary data collections. 

• This system would also provide a framework 
for coordinating existing data collection and 
reporting efforts, providing a more coherent 
and comprehensive view of learning after the 
completion of compulsory education. 

• This system would coincide with and support 
ongoing OECD work (to which NCES contrib- 
utes) to develop international indicators that 
describe lifelong learning and continuing edu- 
cation and training. 

• This system would make U.S. data collections 
more comparable to those in many other de- 
veloped countries, where national surveys of 
continuing education are routinely adminis- 
tered (e.g., Canada, Finland, United Kingdom, 
Germany, Switzerland, Sweden). 

Cons 

• This data collection system does not fit within 
the narrow (traditional) interpretation of NCES’ 
mission, and thus could be viewed as an in- 
appropriate use of funds. 

• NCES staff and fiscal resources are tight; there 
are currently no resources available for ex- 
panding NCES’ mission. 

Taking these factors together, the task force de- 
cided that the “pros" outweigh the “cons.” Regard- 
ing the first “con,” as discussed above, we be- 
lieve the Center can and should argue for a larger, 
broader interpretation of its data collection mis- 
sion. We do acknowledge the very real constraint 
of tight resources, and recognize that any actions 
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NCES takes in this area are likely to be limited by 
this constraint. However, we do not believe the 
Center’s vision should be limited by a lack of re- 
sources; we propose a vision of change in this 
report that we hope can serve as an impetus for 
expanding resources that will keep the Center at 
the forefront of policy-relevant data collection. 

By accepting the value of creating a lifelong learn- 
ing data collection and reporting system, we re- 
ject the “status quo.” That is, we do not believe 
that current efforts within the Center are sufficient 
to meet the existing need for information on life- 
long learning. Current efforts are insufficient for 
two reasons. First, too many important issues are 
left unaddressed by our current data collection 
system — issues such as the prevalence of non- 
traditional forms of adult learning, the decision- 
making process and incentives that affect partici- 
pation in adult learning, and the role of the work- 
place in determining the need for adult learning. 
Second, NCES’ existing data collection efforts fail 
to provide a coherent system of data collection and 
reporting. By continuing these efforts in an unco- 
ordinated fashion, the Center will fail to capitalize 
on its existing resources, and will miss an invalu- 
able opportunity to proactively advance its mission 
and its service to the American public and policy- 
makers. 

Perhaps more importantly, we believe that NCES 
should adopt longer-term goals that recognize the 
globalization of economic and social systems and 
the more integrated perspective on education in- 
herent in the term "lifelong learning.” With this in 
mind, our recommendations are predicated on the 
following implicit goals. 

• Short-term goal: Develop a comprehensive 
data collection and reporting system that moni- 
tors adult education and training. 

Long-term goals: 

• Develop a Center-wide, integrated, comprehen- 
sive data collection and reporting system that 
monitors lifelong learning as broadly conceived, 
from early childhood through adult learning. 

• Coordinate NCES’ national data collection ef- 
forts with their international counterparts, so 
that national collections become expanded or 
revised versions of their international collections 
rather than independent efforts. 



Proposed Data Framework 

Although our task force focused on identifying data 
gaps rather than data availability, it is clear that 
existing data within NCES, especially if supple- 
mented with DoL and Census data, provide a 
strong foundation for building a lifelong learning 
monitoring system. We propose that the core of 
such a system be composed of three existing sur- 
veys — the NHES/AE, NAAL, and ILSS. Together, 
these data sources provide extensive information 
for describing adults’ participation in learning ac- 
tivities, their skill levels, and an international con- 
text in which to interpret findings. For supplemen- 
tal information on work-related and employer-spon- 
sored training, these core data sources should be 
supplemented with data from SIPP and NLS-Y. 
Historically, NCES has seldom used these data 
sets, but they are potentially rich sources of infor- 
mation on lifelong learning that should not be ig- 
nored. For supplemental information on the post- 
secondary sector of adult learning, the core data 
sets would be supplemented with the NPSAS and 
its follow-ups, NSOPF, and the PEQIS system. 

Although all of these surveys are useful in their 
current form, they require expansion and revision 
in order to provide more complete, reliable data. 
As discussed below, some of the data gaps iden- 
tified in section 2 should be targeted for immedi- 
ate attention. Since each of the core surveys 
(NHES/AE, NAAL, ILSS) is currently in the pro- 
cess of being revised, we urge NCES manage- 
ment to act quickly to support a coordinated effort 
to revise these surveys according to the priorities 
outlined in this report. 

In addition, information from the three core data 
sources should be linked to labor market data. 
Exactly how to do this is as yet unclear, but our 
work suggests that the 0*NET and CPS data on 
job skills and employment patterns may provide 
viable sources for linkage. Currently, we know too 
little about the structure and operations of these 
data collections, or about cost-effective means for 
coding occupation data at a detailed level, to make 
specific recommendations on how to use occupa- 
tional skills data. As discussed elsewhere in this 
report, we need to learn more about how to use 
such information. 

One source of information would be incomplete 
in the data system outlined above — information 
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from providers. To remedy this situation, we pro- 
pose that NCES consider the addition of an em- 
ployer survey, to be administered in collaboration 
with DoL, and a modification of the PEQIS or 
IPEDS system, to permit the collection of more 
detailed information on adult learning within post- 
secondary institutions. 

Options for Developing a 
Lifelong Learning System 

Our work over the past year suggests a number 
of activities that NCES should pursue to develop 
a comprehensive lifelong learning data collection 
and reporting system. For strategic planning pur- 
poses, it seemed best to propose a wide range 
of activities, so that each activity can be consid- 
ered both on its own and within the larger frame- 
work of potential activities. We do not expect that 
all of these options can be implemented at the 
current time, or even in the near future. However, 
we believe that all options should eventually be 
pursued if NCES is to fully implement a system- 
atic and comprehensive lifelong learning data 
collection and reporting system. 

Proposed activities are listed below, in the fol- 
lowing order: activities that (1) can be conducted 
with existing data collection sources, (2) would 
require an expansion or revision of existing data 
collection sources, (3) would create new data 
collection sources, and (4) could be conducted 
independently of the level of data collection 
sources. (Note that we did not rank activities in 
terms of required resources.) After delineating the 
full range of proposed activities, we focus on those 
activities that seem most appropriate for the Cen- 
ter to pursue first. 

Step 1. Work With Existing 
Data Collection Sources 

A wide range of data sources on lifelong learning 
currently exist that have not been fully exploited. 
One simple option is to better utilize these exist- 
ing data sources. 

• Consolidate all currently available information 
on lifelong learning into a compendium report, 
using an indicators framework. 

• Expand and coordinate the existing publica- 
tion and reporting activities that use current 
data collection sources. 
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Step 2. Expand Capabilities of 
Existing Data Collection Sources 

As discussed in section 2, current data are in 
some cases incomplete or of questionable reli- 
ability. Efforts should be made to fill these gaps 
in existing data sources. 

• Revise relevant NCES surveys, as opportu- 
nity permits, to better address lifelong learn- 
ing issues. (This activity is ongoing for 
NHES/AE, in conjunction with the task force’s 
work.) 

• Work with other relevant federal agencies 
(Census Bureau, BLS) to modify their surveys 
to provide better data on lifelong learning is- 
sues. 

• Conduct developmental work to resolve the 
conceptual and measurement issues identi- 
fied in lifelong learning. (As discussed below, 
this proposal calls for a more intensive devel- 
opmental activity than that currently conducted 
as part of survey development efforts.) 

Step 3. Develop New 
Data Collection Sources 

Even after filling gaps in existing data sources, 
some gaps will remain that can be filled only with 
new data collection efforts. 

• Conduct PEQIS surveys to fill data gaps that 
can be addressed through this data collection 
method (e.g., contract training, postsecond- 
ary services and accommodations). 

• Open a dialogue with DoL about a possible 
collaborative effort to develop and administer 
an employer survey, addressing issues related 
to employer offerings, support and incentives 
for learning (particularly for specific groups of 
workers), reliance on formal versus informal 
learning, and skill and learning demands. 

Supporting Activities 

We also propose a number of additional activi- 
ties to help institutionalize lifelong learning efforts 
within NCES. These activities could be imple- 
mented at various stages, although most are best 
pursued as part of an expansion of data collec- 
tion resources (steps 2 and 3 above): 

• Develop a lifelong learning publications and 
analysis plan. 
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• Open a dialogue with OERI to develop a co- 
ordinated approach to data collection and re- 
search on lifelong learning issues. 

• Work with DoL to determine how to cost-ef- 
fectively produce and use detailed occupa- 
tional data, and how to link such information 
to other data collection sources, including the 
0*NET. 

• Develop new indicators of lifelong learning for 
inclusion in current NCES compendium pub- 
lications (i.e., Condition of Education, Digest 
of Education Statistics, Vocational Education 
in the United States). 

• Encourage institutional/management-level 
support for lifelong learning by holding man- 
agement meetings at which program manag- 
ers and division directors debate the issues 
raised in this report. 

• Institute a regular NCES seminar series (on 
varying topics) to help build internal aware- 
ness and support for lifelong learning (and 
other) issues, and to provide an opportunity 
for collegial interactions and information shar- 
ing. 

• Sponsor professional conference presenta- 
tions and panels on lifelong learning issues. 

• Implement a policy to make all data collection 
instruments easily accessible in a user-friendly 
format (e.g., survey “versions” of CATI proto- 
cols), to allow analysts to more easily deter- 
mine the availability of lifelong learning (as well 
as other) data on each instrument. 

Recommended First Steps 

As a first step, two fundamental problems need 
to be attacked: (1) the under-use of existing data 
on adult learning, and (2) the limitations of exist- 
ing data. NCES should act to simultaneously rem- 
edy these problems by improving the data report- 
ing system and the data collection system for life- 
long learning. Specifically, the first steps should 
be to (1) publish a compendium report on adult 
learning, and (2) revise existing surveys to col- 
lect better data on adult learning. To support fu- 
ture data collection and reporting, we also pro- 
pose the implementation of a process for inten- 
sive item development work, independent of in- 
dividual survey redesign efforts. 



Improve Data Reporting. As mentioned above, 
NCES currently collects much data relevant to 
adult learning. A number of reports have been 
published using these data. From a lifelong learn- 
ing perspective, however, the independent, nar- 
rowly focused structure of these reports fails to 
fully realize the potential of existing NCES data. 
The Center currently has the capability, through 
a coordinated analysis and synthesis of existing 
data, to provide a comprehensive overview of 
adult learning similar to that provided in reports 
on other education-related topics such as the well- 
being of children, high school dropouts, vocational 
education, and student achievement. We recom- 
mend that NCES conduct a more systematic and 
comprehensive analysis of existing data on life- 
long learning, and that the results of this analysis 
be published in a compendium report. 

This compendium report will serve many pur- 
poses. First, it will more broadly publicize NCES 
data and their range of potential uses. Second, it 
will allow us to explore the feasibility of using sur- 
veys from DoL and the Census Bureau (e.g., 
NLS-Y, SIPP) as part of a lifelong learning data 
collection system. Third, it will indicate more 
clearly (than our work could) the potential and 
limitations of current data on lifelong learning. For 
example, analyses conducted for this report 
should reveal the comparability of findings across 
data sets. Fourth, response to the publication will 
provide feedback on (1) the level of interest in 
this topic in general, and (2) the specific areas of 
lifelong learning for which the public and policy- 
makers would most like to have more informa- 
tion. This publication also could serve as the 
springboard for other recommended activities 
listed above, such as the development of a publi- 
cations plan, coordination with OERI, and indi- 
cators development. Because of the amount that 
can be learned during the production of this com- 
pendium report, we recommend that the report 
be produced in-house, with analytic support from 
one or more contractors. 

Improve Data Collection. A compendium report 
can also inform decisions on how to revise exist- 
ing data collection instruments. But the task force 
feels that data revision activities should not wait 
for the completion of that report. We are confi- 
dent that we know enough now to allow us to 
make significant improvement in existing surveys’ 
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ability to address lifelong learning issues. Further, 
some of the most critical surveys for adult learn- 
ing are being redesigned at present or in the near 
future (e.g., NHES/AE, ILSS, NAAL, NELS, 
NPSAS). Because the task force includes repre- 
sentatives from most of these survey efforts, our 
work has been informally coordinated with these 
redesign efforts during the past year. However, 
these efforts need to be officially sanctioned and 
supported, particularly if they are to continue af- 
ter the task force has disbanded. 

A systematic, coordinated effort to expand and 
refine lifelong learning data in NCES survey re- 
designs has the potential to provide a large pay- 
off in data and reporting capabilities with a mini- 
mum of additional resource investment. This sur- 
vey revision effort would address one of the most 
fundamental problems in existing lifelong learn- 
ing data that was identified in the task force’s 
work — the large number of “data gaps." Some of 
these gaps can be filled more easily than others, 
and some are more important to fill than others. 
Combining these two criteria, we recommend that 
NCES first focus on expanding or improving ques- 
tions asked about the following issues: adults’ la- 
bor market status, satisfaction with existing learn- 
ing opportunities, intensity (or volume) of learn- 
ing, intended and actual benefits of participation, 
specific skills sought through learning, induce- 
ments and constraints affecting participation, al- 
ternative credentialing and delivery methods, 
availability of employer support, informal work- 
related education, and noncredit coursetaking. 
Investigation into the feasibility of using 0*NET 
data to track workforce skill demands should also 
be pursued as part of initial efforts to improve the 
existing data collection system. This effort would 
also help fill important data gaps. 

Item Development. We also propose that NCES 
conduct developmental work to resolve concep- 
tual and measurement issues in lifelong learning 
data. This type of work is routinely done as part 
of NCES efforts to design or redesign a data col- 
lection instrument. These efforts tend to be lim- 
ited, however, as they are typically conducted 
within the strict time and fiscal constraints of a 
larger survey effort. For issues that can be re- 
solved fairly easily, this procedure works well. But 
for issues that are not easily resolved or that cut 
across data collection efforts, this procedure can 
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be inadequate. These more complex issues call 
for a more focused, intensive developmental ac- 
tivity undertaken independently of a survey re- 
design. The results of such studies would even- 
tually be used in survey redesigns, but the stud- 
ies themselves would not be constrained by a 
particular survey’s schedule (or budget). 

As an area of data collection, lifelong learning 
appears to have a relatively large number of diffi- 
cult, unresolved conceptual issues. This occurs 
partly because lifelong learning is a relatively new 
field of inquiry, and partly because it is a diverse, 
multidimensional, often unsystematic enterprise. 
Thus, while the recommendation for more inten- 
sive developmental work could apply to any NCES 
data collection effort, we feel that this recommen- 
dation is particularly relevant to efforts related to 
lifelong learning. Issues in lifelong learning that 
could benefit from this type of developmental work 
include: determining the types of learning and 
skills that are important to monitor; determining 
which informal learning activities should be moni- 
tored and how to do so; collecting better data on 
providers and provider collaborations; modeling 
the participation decision-making process; collect- 
ing more valid learning intensity measures; de- 
termining the types of adult services and ac- 
commodations that should be monitored; and 
measuring adult learning costs. It should be noted 
that the resolution of these issues will often be of 
benefit to other data collection systems in addi- 
tion to lifelong learning; for example, models of 
the decision-making process might be of use on 
postsecondary surveys. 

Because of the value of this developmental ac- 
tivity both for lifelong learning and other NCES 
data collection efforts, we recommend that this 
activity also be pursued in conjunction with or 
immediately following the development of a com- 
pendium report and the revision of existing sur- 
veys. The completion of the latter two activities 
could provide further insight into which issues are 
most worth an intensive developmental effort. 

Organizational 
Implementation Options 

How should this work be carried out, both in the 
short-term, while a lifelong learning data collec- 
tion and reporting system is underdevelopment, 
and in the long-term, when this system becomes 
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fully functional and institutionalized within NCES? 
We propose some organizational options below, 
based largely on existing models within the Cen- 
ter. 

The DOVE Model 

The Center’s data collection and reporting sys- 
tem for vocational education (Data on Vocational 
Education, or DOVE system) serves as one model 
for how to organize a lifelong learning data col- 
lection system. Within NCES, lifelong learning is 
similar to vocational education in many ways, 
most notably in its reliance on data from a variety 
of surveys that are administered by different divi- 
sions (see figure 3.1). 

The DOVE system is coordinated by one NCES 
staff-person, located in the Data Development 
Program within the Early Childhood, International, 
and Cross-Cutting Studies Division. This person 
develops the vocational education publications 
plan, oversees a Vocational Education Technical 
Review Panel (TRP), monitors contract work on 
vocational education reports, and attempts to 
improve and expand NCES data on vocational 
education through informal representation on the 
TRPs for relevant data collection efforts. This 
model works reasonably well except for the last 
task. Both the current and past DOVE coordina- 
tors, as well as other NCES staff, have noted that 
the informal reliance on one person to influence 
multiple data collections is time-consuming and 
not very effective. 

Would this model work for lifelong learning? We 
think that the problems with the current DOVE 
system would only be exaggerated in lifelong 
learning, where the limits of the existing data col- 
lection system are more extensive than they are 
in vocational education. Lifelong learning is an 
emerging area in which much work needs to be 
done to expand and improve the Center’s data 
collection system. This organizational model is 
ill-suited for an activity that requires such inten- 
sive data development work across multiple sur- 
veys. Thus, we do not recommend the DOVE 
model. 

The Team Approach 

Our Lifelong Learning Task Force provides an- 
other potential model for the organization of life- 
long learning data collection and reporting activi- 
ties. Future lifelong learning work could be con- 



ducted by a similar inter-program team of staff 
members representing the Center’s relevant data 
collection efforts. Our experience on the task force 
has demonstrated the advantages and disadvan- 
tages of the team approach. The advantages are 
many, including a cross-fertilization of ideas, the 
(relative) ease of coordinating efforts on multiple 
surveys, and, at least in theory, more staff re- 
sources. Unfortunately, in our situation staff re- 
sources proved to be very limited, as team mem- 
bers took on this assignment in addition to full- 
time work loads, or with less time assigned to the 
task force than was required. Management sup- 
port was also insufficient in some cases. For ex- 
ample, although the task force had regularly 
scheduled meetings, some program or division 
directors scheduled "impromptu” meetings dur- 
ing our scheduled times, making it impossible for 
task force members to attend meetings. If these 
problems can be avoided, an inter-program team 
could be a sensible approach to accomplishing 
lifelong learning activities. 

Our task force is an inappropriate model in one 
respect. A task force is a temporary group formed 
to solve specific problems in a finite period of time. 
A team, on the other hand, is a long-standing 
group that works on specific problems or tasks 
as part of a continuing effort to manage a pro- 
gram of ongoing work. So, for example, while our 
task force worked for 1 year to develop recom- 
mendations on lifelong learning data collection 
activities, a lifelong learning team would have 
long-term responsibilities for ensuring that an ef- 
fective data collection and reporting system was 
developed, implemented, and improved as nec- 
essary. 

The key to the team approach is to ensure that 
the team has effective leadership, adequate staff 
and budget resources, and a clearly defined mis- 
sion and tasks. For the Lifelong Learning Team, 
we can envision a team that has a permanent 
team leader, or in which the leadership changes 
depending on the task at hand. For example, for 
NHES/AE item development, a team member 
who works on that survey could lead those ac- 
tivities, while a different team member could lead 
efforts to develop a report on work-related learn- 
ing that draws from a range of surveys. Team 
members should include NCES staff who have 
major responsibility for each survey that contains 
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data on lifelong learning. At the least, this would 
include staff from NPSAS, NHES/AE, and the 
adult literacy assessments. Budget and support 
staff should be provided as appropriate for team 
tasks, which should include the specific tasks of 
item development, analysis design, and report 
planning, writing, and review. 

We also believe that work on lifelong learning 
should become institutionalized within the Cen- 
ter, as part of its divisional and program struc- 
ture. Our next two recommendations focus on 
these organizational options. 

Creating New Program Area(s) 

The only way to fully integrate a data collection 
program into NCES is to make it an integral part 
of the Center’s organizational structure. Specifi- 
cally, work on lifelong learning needs to be orga- 
nized within a program area. This work could be 
organized within an existing program, or in a new 
program (or programs). If this work were assigned 
to an existing program area, the Interagency and 
Household Studies Program would be the most 
logical choice, as the NHES/AE is likely to be 
NCES’ central data collection instrument for life- 
. long learning. The problem with this approach is 
the obvious limitation of having staff from only 
one survey involved in work that requires using 
and modifying many surveys across the Center. 

The creation of one or more new program areas 
could help alleviate the coordination problem and 
would further integrate lifelong learning within 
NCES. We propose two possible alternatives 
along these lines in figures 3.2 and 3.3. In these 
models, surveys with data relevant to adult learn- 
ing would be shifted among programs; there 
would be no net change in the number of pro- 
grams. 

Under the model depicted in figure 3.2, the na- 
tional and international adult literacy surveys 
would be combined into an “Adult Literacy As- 
sessment” program within the Assessment Divi- 
sion. The newly named Lifelong Learning, Inter- 
national, and Cross-Cutting Studies Division 
would lose a program area, as “Early Childhood” 
and “Interagency and Household Studies” pro- 
grams would be combined into the “Early Child- 
hood and Adult Learning” Program, overseeing 
the ECLS and the NHES early childhood and 
adult education components. The lifelong iearn- 
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ing data collection system would largely be en- 
compassed (except for its postsecondary com- 
ponents) by the new “Early Childhood and Adult 
Learning” and “Adult Literacy Assessment” pro- 
gram areas. 

The model depicted in figure 3.3 requires a slightly 
more extensive reorganization, but has the ad- 
vantage of placing the program areas with the 
greatest relevance to lifelong learning within one 
division. In this model, “Early Childhood and Adult 
Learning" becomes a single program area, as in 
the previous model, but the international student 
assessments and adult literacy assessments are 
essentially switched. Thus, the K-1 2 international 
student assessments are part of the Assessment 
Division, and the adult literacy assessments form 
an “Adult Literacy" Program in what is currently 
called the Early Childhood, International, and 
Cross-Cutting Division (to be renamed the Life- 
long Learning and Cross-Cutting Studies Divi- 
sion). 

Because surveys containing lifelong learning data 
would cut across program areas under either of 
these models, we propose that even with the in- 
stitutionalization of lifelong learning within an ex- 
isting or new program area, the inter-program 
team approach discussed above be maintained; 
this is the only way to ensure effective coordina- 
tion across relevant program areas. 

The reorganizations listed in figures 3.2 and 3.3 
seem quite feasible under current conditions. We 
strongly encourage NCES management to con- 
sider implementing changes along these lines in 
the near future. These reorganizations also move 
NCES closer to the long-term goal of an organi- 
zation centered around the lifelong learning con- 
cept and an international framework. We now turn 
to this longer-term goal. 

Creating a New Division 
(Reorganization) 

NCES has just been through a major reorganiza- 
tion, and no one in the Center is anxious to go 
through that process again. We would not rec- 
ommend that the Center undergo a reorganiza- 
tion anytime soon. Nonetheless, the Center’s cur- 
rent organization scheme is not likely to be its 
last. Given the inevitability of change, we propose 
a major reorganization that would align the 
Center’s work with the two long-term goals listed 



Lifelong Learning 



above — a Center-wide approach to lifelong learn- 
ing, broadly conceived, and the coordination of 
national and international data collection efforts. 

In this reorganization (see figure 3.4), NCES 
would be divided into five divisions, four of which 
represent the lifelong-learning stages of early 
childhood education, compulsory (K-12) educa- 
tion, postsecondary education, and adult learn- 
ing. (The fifth division encompasses cross- 
cutting activities.) Each division would be respon- 
sible for ensuring that the Center has a compre- 
hensive data collection system for its population, 
or lifelong-learning sector. Surveys would reside 
in programs based on the initial lifelong-learning 
sector to which they apply. For the longitudinal 
surveys, which collect data relevant to more than 
one division (or lifelong-learning sector), the team 
approach would be used to ensure that relevant 
data are collected to address each lifelong-learn- 
ing sector. So, for example, a new NELS data 
collection would be placed in the Compulsory 
Education Division because its data collection 
would begin at that level. The data collection would 
be initially developed by program staff within the 
Compulsory Education Division, but follow-up 
-surveys would be developed by an inter-program 
team consisting of NELS program staff and staff 
from the Postsecondary Education and Adult 
Learning Divisions. (This inter-program team ap- 
proach might also be needed for NHES, to en- 
sure the effective use of household sampling pro- 
cedures.) 

The effect of this organizational structure would 
be that staff could be responsible not just for one 
data collection effort within their division but, 
through their representation on an inter-program 
team, could also have substantive (but not con- 
tractual) responsibilities for data collection efforts 
in other programs or divisions that relate to their 
lifelong learning sector. Someone in the Adult 
Learning Division, for example, would be in charge 
of the NAAL, but could also work with staff on the 
ILSS, NHES/AE, NELS, or NPSAS to ensure that 
data on lifelong learning are collected in the most 
efficient and effective manner across the Center. 

Recommended Organizational Option 

We believe that the major reorganization dis- 
cussed above should be NCES’ ultimate goal; this 
reorganization will allow the Center to collect and 
report data on learning throughout the life cycle, 



reflecting the contemporary view of education as 
a process of “lifelong learning” and the goal of 
creating “a nation of learners.” But for the imme- 
diate future, we propose more limited changes 
that will allow staff to complete the initial activi- 
ties listed above. 

To start, we recommend that NCES officially des- 
ignate an inter-program Lifelong Learning Team 
to work on the concrete tasks of creating a com- 
pendium report and modifying existing data col- 
lection instruments. Why use the team approach 
rather than assigning this work to one or a few 
individuals? We believe that this work will pro- 
ceed most efficiently if it is coordinated across 
the Center and draws upon the combined re- 
sources of staff who are experts on specific sur- 
veys relevant to lifelong learning and/or specific 
issues that pertain to lifelong learning. One rea- 
son the team approach is best for this work is 
that lifelong learning is so broad in scope — in- 
cluding postsecondary education, adult basic 
education, work-related education, and informal 
and “social” activities — and involves so many data 
issues. As discussed above, compared to most 
other areas for which NCES collects data, life- 
long learning includes a relatively large number 
of unresolved conceptual and measurement is- 
sues, draws from a relatively large number of data 
sources, and has a relatively large need for new 
data development. The depth and breadth of is- 
sues encompassed by lifelong learning suggests 
that a coordinated, systematic approach by a 
team of individuals is preferable to reliance on 
the efforts of independent individuals within the 
Center. In sum, if the goal is to develop a com- 
prehensive, coordinated data collection system, 
the work to develop that system should rely on a 
comprehensive, coordinated approach. 

At the conclusion of the team’s initial work (which 
should have a time limit, say of about 3 years), 
the Center should reevaluate the status of its life- 
long learning data collection. Feedback on the 
compendium report and otherteam efforts should 
be used to evaluate the effectiveness of the team 
approach and the feasibility of creating a program 
area in which work on lifelong learning can be 
coordinated and institutionalized. Assuming the 
team approach is found to be fundamentally 
sound, NCES should continue to use this ap- 
proach, with whatever modifications are neces- 
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the eventual goal would be a larger reorganiza- 
tion based on the broad conception of lifelong 
learning. 



sary based on the evaluation. Assuming response 
to the data produced and disseminated by the 
team is positive, the formation of one or more pro- 
gram areas focused on lifelong learning should 
be considered at this time. As discussed above, 



Figure 3.1 — Current NCES Organization (4 Divisions, 14 Programs) 
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Note: Boxes represent program areas, with major data collection efforts listed below the dotted line. Data collections listed 
in bold are those that would contribute to a lifelong learning data collection system. 
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Figure 3.2 — Creating New Program Areas, Model 1 (4 Divisions, 14 Programs) 
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Note: Boxes represent program areas, with major data collection efforts listed below the dotted line. Data collections listed 
in bold are those that would contribute to a lifelong learning data collection system. Shaded boxes represent new or revised 
program areas. 



The changes proposed in this figure would affect 
two divisions, as follows: 

• The Assessment Division would gain a pro- 
gram area. The new program, “Adult Literacy 
Assessment,” would be responsible for the 
NAAL and (moved from the “International 
Studies” Program in the Early Childhood, In- 
ternational, and Cross-Cutting Studies Divi- 
sion) the IALS/ILSS. 



• The newly named Lifelong Learning, Interna- 
tional, and Cross-Cutting Studies Division 
would be reorganized from five programs to 
four, as “Early Childhood” would be combined 
with the “Interagency and Household Studies” 
Program to form an “Early Childhood and Adult 
Learning” Program. 
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Figure 3.3 — Creating New Program Areas, Model 2 (4 Divisions, 14 Programs) 
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Note: Boxes represent program areas, with major data collection efforts listed below the dotted line. Data collections listed 
in bold are those that would contribute to a lifelong learning data collection system. Shaded boxes represent new or revised 
program areas. 



The changes proposed in this figure would affect 
two divisions, as follows: 

• The newly named Elementary/Secondary As- 
sessment Division would lose responsibility for 
the NAAL, but would gain an “International 
Student Assessment” Program to operate the 
TIMSS, PISA, and other international student 
assessment projects. 

• The newly named Lifelong Learning and 
Cross-Cutting Studies Division would be re- 



organized from five programs to four, as “Early 
Childhood” would be combined with the “In- 
teragency and Household Studies” Program 
to form an “Early Childhood and Adult Learn- 
ing” Program. The “International Studies” Pro- 
gram would be replaced by an “Adult Literacy” 
Program, with oversight for NAAL (moved from 
the Assessment Division) and ILSS. 



40 

o 

ERIC 



46 



Lifelong Learning 



Figure 3.4 — Creating a New Division: Reorganization (5 Divisions, 14 Programs) 
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Note: Boxes represent program areas, with major data collection efforts listed below the dotted line. Data collections listed 
in bold are those that would contribute to a lifelong learning data collection system. 



This reorganization would affect every current 
division except for the Postsecondary Studies Di- 
vision: 

• The Assessment Division would become part 
of the Compulsory Education Division. 

• The Elementary, Secondary, and Library Stud- 
ies Division would gain a “Student Assess- 
ment” Program (the original Assessment Di- 
vision); the elementary/secondary and library 
cooperative systems would be combined into 
one program. 



• The Early Childhood, International, and Cross- 
Cutting Studies Division would be divided into 
three divisions — “Early Childhood Education,” 
“Adult Learning,” and “Cross-Cutting Studies.” 
TIMSS would be replaced by NAAL, as the 
former moves to the Compulsory Education 
Division and the latter to the Adult Learning 
Division. 
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ENDNOTES 



^he six issue areas, listed in table 1 .2, were developed through group discussion at task force meetings. We attempted to 
divide lifelong learning into areas that covered major topics of policy concern, and that were relatively independent of 
each other. 

2 Appendix H in Volume II provides more detailed information on the data available to address each issue; this appendix 
summarizes task force meetings during which data availability was discussed. The numbering system used in this section 
of the report matches that used in appendix H (and in appendix E, the complete list of prioritized issues). 

3 The U.S. datasets referred to are typically the most recent editions of these surveys as of summer 1999. However, refer- 
ences to NHES/AE refer to the 1 995 version of that survey unless otherwise noted, and references to the NES refer to the 
1 994 version unless otherwise noted. For the ILSS, we used as our reference the July 1 999 version of the “participation in 
education and learning” module recommended by OECD’s Network B. For the NELS, we used the field-test version of the 
NELS fourth follow-up survey. 
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Listing of NCES Working Papers to Date 



Working papers can be downloaded as pdf files from the NCES Electronic Catalog 
(http://nces.ed.gov/pubsearch/) . You can also contact Sheilah Jupiter at (202) 502-7444 
(sheilah_jupiter@ed.gov) if you are interested in any of the following papers. 

Listing of NCES Working Papers by Program Area 

No. Title NCES contact 



Baccalaureate and Beyond (B&B) 

98-15 Development of a Prototype System for Accessing Linked NCES Data 



Steven Kaufman 



Beginning Postsecondary Students (BPS) Longitudinal Study 

98-1 1 Beginning Postsecondary Students Longitudinal Study First Follow-up (BPS:96-98) Field Aurora D’Amico 
Test Report 

98- 1 5 Development of a Prototype System for Accessing Linked NCES Data Steven Kaufman 

1 999-1 5 Projected Postsecondary Outcomes of 1 992 High School Graduates Aurora D’Amico 



Common Core of Data (CCD) 



95- 12 Rural Education Data User’s Guide 

96- 1 9 Assessment and Analysis of School-Level Expenditures 

97- 1 5 Customer Service Survey: Common Core of Data Coordinators 

97- 43 Measuring Inflation in Public School Costs 

98- 1 5 Development of a Prototype System for Accessing Linked NCES Data 

1 999- 03 Evaluation of the 1996-97 Nonfiscal Common Core of Data Surveys Data Collection, 

Processing, and Editing Cycle 

2000- 12 Coverage Evaluation of the 1 994-95 Common Core of Data: Public 

Elementary/Secondary School Universe Survey 

2000-13 Non-professional Staff in the Schools and Staffing Survey (SASS) and Common Core of 
Data (CCD) 



Samuel Peng 
William J. Fowler, Jr. 
Lee Hoffman 
William J. Fowler, Jr. 
Steven Kaufman 
Beth Young 

Beth Young 

Kerry Gruber 



Data Development 

2000- 1 6a Lifelong Learning NCES Task Force: Final Report Volume I Lisa Hudson 

2000- 1 6b Lifelong Learning NCES Task Force: Final Report Volume II Lisa Hudson 

Decennial Census School District Project 

95- 12 Rural Education Data User’s Guide Samuel Peng 

96- 04 Census Mapping Project/School District Data Book Tai Phan 

98-07 Decennial Census School District Project Planning Report Tai Phan 



Early Childhood Longitudinal Study (ECLS) 

96-08 How Accurate are Teacher Judgments of Students’ Academic Performance? Jerry West 

96- 1 8 Assessment of Social Competence, Adaptive Behaviors, and Approaches to Learning with Jerry West 

Young Children 

97- 24 Formulating a Design for the ECLS: A Review of Longitudinal Studies Jerry West 

97-36 Measuring the Quality of Program Environments in Head Start and Other Early Childhood Jerry West 

Programs: A Review and Recommendations for Future Research 

1 999- 01 A Birth Cohort Study: Conceptual and Design Considerations and Rationale Jerry West 

2000- 04 Selected Papers on Education Surveys: Papers Presented at the 1998 and 1999 ASA and Dan Kasprzyk 

1999 AAPOR Meetings 



Education 

94-05 

96- 19 

97- 43 

98- 04 
1999-16 



Finance Statistics Center (EDFIN) 

Cost-of-Education Differentials Across the States 
Assessment and Analysis of School-Level Expenditures 
Measuring Inflation in Public School Costs 
Geographic Variations in Public Schools’ Costs 

Measuring Resources in Education: From Accounting to the Resource Cost Model 
Approach 



William J. Fowler, Jr. 
William J. Fowler, Jr. 
William J. Fowler, Jr. 
William J. Fowler, Jr. 
William J. Fowler, Jr. 



No. 



Title 



NCES contact 



High School and Beyond (HS&B) 

95-12 Rural Education Data User’s Guide 
1 999-05 Procedures Guide for Transcript Studies 

1 999-06 1 998 Revision of the Secondary School Taxonomy 

HS Transcript Studies 

1 999-05 Procedures Guide for Transcript Studies 

1 999- 06 1 998 Revision of the Secondary School Taxonomy 

International Adult Literacy Survey (IALS) 

97-33 Adult Literacy: An International Perspective 

Integrated Postsecondary Education Data System (IPEDS) 

97- 27 Pilot Test of IPEDS Finance Survey 

98- 15 Development of a Prototype System for Accessing Linked NCES Data 

2000- 14 IPEDS Finance Data Comparisons Under the 1997 Financial Accounting Standards for 

Private, Not-for-Profit Institutes: A Concept Paper 



National Assessment of Adult Literacy (NAAL) 

98-17 Developing the National Assessment of Adult Literacy: Recommendations from 
Stakeholders 

1 999-09a 1 992 National Adult Literacy Survey: An Overview 

1 999-09b 1 992 National Adult Literacy Survey: Sample Design 

1 999-09c 1 992 National Adult Literacy Survey: Weighting and Population Estimates 

1 999-09d 1 992 National Adult Literacy Survey: Development of the Survey Instruments 

1999-09e 1992 National Adult Literacy Survey: Scaling and Proficiency Estimates 

1 999-09f 1 992 National Adult Literacy Survey: Interpreting the Adult Literacy Scales and Literacy 

Levels 

1 999- 09g 1 992 National Adult Literacy Survey: Literacy Levels and the Response Probability 

Convention 

2000- 05 Secondary Statistical Modeling With the National Assessment of Adult Literacy: 

Implications for the Design of the Background Questionnaire 
2000-06 Using Telephone and Mail Surveys as a Supplement or Alternative to Door-to-Door 
Surveys in the Assessment of Adult Literacy 

2000-07 “How Much Literacy is Enough?” Issues in Defining and Reporting Performance 
Standards for the National Assessment of Adult Literacy 
2000-08 Evaluation of the 1 992 NALS Background Survey Questionnaire: An Analysis of Uses 
with Recommendations for Revisions 
2000-09 Demographic Changes and Literacy Development in a Decade 



National Assessment of Educational Progress (NAEP) 

95-12 Rural Education Data User’s Guide 

97-29 Can State Assessment Data be Used to Reduce State NAEP Sample Sizes? 

97-30 ACT’s NAEP Redesign Project: Assessment Design is the Key to Useful and Stable 



Assessment Results 

97-31 NAEP Reconfigured: An Integrated Redesign of the National Assessment of Educational 
Progress 

97-32 Innovative Solutions to Intractable Large Scale Assessment (Problem 2: Background 
Questionnaires) 

97-37 Optimal Rating Procedures and Methodology for NAEP Open-ended Items 

97- 44 Development of a SASS 1 993-94 School-Level Student Achievement Subfile: Using 

State Assessments and State NAEP, Feasibility Study 

98- 15 Development of a Prototype System for Accessing Linked NCES Data 

1999-05 Procedures Guide for Transcript Studies 

1 999-06 1 998 Revision of the Secondary School Taxonomy 



National Education Longitudinal Study of 1988 (NELS:88) 

95-04 National Education Longitudinal Study of 1988: Second Follow-up Questionnaire Content 
Areas and Research Issues 
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Samuel Peng 
Dawn Nelson 
Dawn Nelson 



Dawn Nelson 
Dawn Nelson 



Marilyn Binkley 



Peter Stowe 
Steven Kaufman 
Peter Stowe 



Sheida White 

Alex Sedlacek 
Alex Sedlacek 
Alex Sedlacek 
Alex Sedlacek 
Alex Sedlacek 
Alex Sedlacek 

Alex Sedlacek 

Sheida White 

Sheida White 

Sheida White 

Sheida White 

Sheida White 



Samuel Peng 
Steven Gorman 
Steven Gorman 

Steven Gorman 

Steven Gorman 

Steven Gorman 
Michael Ross 

Steven Kaufman 
Dawn Nelson 
Dawn Nelson 



Jeffrey Owings 



No. 



Title 



NCES contact 



95-05 National Education Longitudinal Study of 1988: Conducting Trend Analyses of NLS-72, 
HS&B, and NELS:88 Seniors 

95-06 National Education Longitudinal Study of 1988: Conducting Cross-Cohort Comparisons 
Using HS&B, NAEP, and NELS:88 Academic Transcript Data 

95-07 National Education Longitudinal Study of 1988: Conducting Trend Analyses HS&B and 
NELS:88 Sophomore Cohort Dropouts 

95-12 Rural Education Data User’s Guide 

95- 14 Empirical Evaluation of Social, Psychological, & Educational Construct Variables Used 

in NCES Surveys 

96- 03 National Education Longitudinal Study of 1988 (NELS:88) Research Framework and 

Issues 

98-06 National Education Longitudinal Study of 1988 (NELS:88) Base Year through Second 
Follow-Up: Final Methodology Report 

98-09 High School Curriculum Structure: Effects on Coursetaking and Achievement in 

Mathematics for High School Graduates — An Examination of Data from the National 
Education Longitudinal Study of 1988 

98-15 Development of a Prototype System for Accessing Linked NCES Data 
1 999-05 Procedures Guide for Transcript Studies 
1 999-06 1 998 Revision of the Secondary School Taxonomy 

1 999-1 5 Projected Postsecondary Outcomes of 1 992 High School Graduates 



National Household Education Survey (NHES) 

95- 12 Rural Education Data User’s Guide 

96- 13 Estimation of Response Bias in the NHES:95 Adult Education Survey 

96-14 The 1995 National Household Education Survey: Reinterview Results for the Adult 
Education Component 

96-20 1991 National Household Education Survey (NHES:91) Questionnaires: Screener, Early 

Childhood Education, and Adult Education 

96-21 1993 National Household Education Survey (NHES:93) Questionnaires: Screener, School 

Readiness, and School Safety and Discipline 

96-22 1995 National Household Education Survey (NHES:95) Questionnaires: Screener, Early 

Childhood Program Participation, and Adult Education 
96-29 Undercoverage Bias in Estimates of Characteristics of Adults and 0- to 2-Year-Olds in the 

1995 National Household Education Survey (NHES:95) 

96- 30 Comparison of Estimates from the 1995 National Household Education Survey 

(NHES:95) 

97- 02 Telephone Coverage Bias and Recorded Interviews in the 1 993 National Household 

Education Survey (NHES:93) 

97-03 1991 and 1995 National Household Education Survey Questionnaires: NHES:91 Screener, 

NHES:91 Adult Education, NHES:95 Basic Screener, and NHES:95 Adult Education 

97-04 Design, Data Collection, Monitoring, Interview Administration Time, and Data Editing in 
the 1993 National Household Education Survey (NHES:93) 

97-05 Unit and Item Response, Weighting, and Imputation Procedures in the 1993 National 
Household Education Survey (NHES:93) 

97-06 Unit and Item Response, Weighting, and Imputation Procedures in the 1 995 National 
Household Education Survey (NHES:95) 

97-08 Design, Data Collection, Interview Timing, and Data Editing in the 1 995 National 
Household Education Survey 

97-19 National Household Education Survey of 1 995: Adult Education Course Coding Manual 

97-20 National Household Education Survey of 1995: Adult Education Course Code Merge 

Files User’s Guide 

97-25 1 996 National Household Education Survey (NHES:96) Questionnaires: 

Screener/Household and Library, Parent and Family Involvement in Education and 
Civic Involvement, Youth Civic Involvement, and Adult Civic Involvement 
97-28 Comparison of Estimates in the 1996 National Household Education Survey 
97-34 Comparison of Estimates from the 1993 National Household Education Survey 
97-35 Design, Data Collection, Interview Administration Time, and Data Editing in the 1 996 
National Household Education Survey 

97-38 Reinterview Results for the Parent and Youth Components of the 1 996 National 
Household Education Survey 



Jeffrey Owings 

Jeffrey Owings 

Jeffrey Owings 

Samuel Peng 
Samuel Peng 

Jeffrey Owings 

Ralph Lee 

Jeffrey Owings 

Steven Kaufman 
Dawn Nelson 
Dawn Nelson 
Aurora D’Amico 

Samuel Peng 
Steven Kaufman 
Steven Kaufman 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Kathryn Chandler 

Peter Stowe 
Peter Stowe 

Kathryn Chandler 

Kathryn Chandler 
Kathryn Chandler 
Kathryn Chandler 

Kathryn Chandler 




51 



No. 



Title 



NCES contact 



97-39 Undercoverage Bias in Estimates of Characteristics of Households and Adults in the 1996 
National Household Education Survey 

97- 40 Unit and Item Response Rates, Weighting, and Imputation Procedures in the 1 996 

National Household Education Survey 

98- 03 Adult Education in the 1 990s: A Report on the 1991 National Household Education 

Survey 

98-1 0 Adult Education Participation Decisions and Barriers: Review of Conceptual Frameworks 
and Empirical Studies 

National Longitudinal Study of the High School Class of 1972 (NLS-72) 

95- 12 Rural Education Data User’s Guide 

National Postsecondary Student Aid Study (NPSAS) 

96- 17 National Postsecondary Student Aid Study: 1996 Field Test Methodology Report 

National Study of Postsecondary Faculty (NSOPF) 

97- 26 Strategies for Improving Accuracy of Postsecondary Faculty Lists 

98- 15 Development of a Prototype System for Accessing Linked NCES Data 

2000-01 1 999 National Study of Postsecondary Faculty (NSOPF:99) Field Test Report 

Postsecondary Education Descriptive Analysis Reports (PEDAR) 

2000-1 1 Financial Aid Profile of Graduate Students in Science and Engineering 

Private School Universe Survey (PSS) 

95-16 Intersurvey Consistency in NCES Private School Surveys 

95- 1 7 Estimates of Expenditures for Private K-l 2 Schools 

96- 16 Strategies for Collecting Finance Data from Private Schools 

96-26 Improving the Coverage of Private Elementary-Secondary Schools 

96- 27 Intersurvey Consistency in NCES Private School Surveys for 1993-94 

97- 07 The Determinants of Per-Pupil Expenditures in Private Elementary and Secondary 

Schools: An Exploratory Analysis 

97- 22 Collection of Private School Finance Data: Development of a Questionnaire 

98- 15 Development of a Prototype System for Accessing Linked NCES Data 

2000-04 Selected Papers on Education Surveys: Papers Presented at the 1 998 and 1 999 ASA and 
1 999 AAPOR Meetings 

2000-1 5 Feasibility Report: School-Level Finance Pretest, Private School Questionnaire 

Recent College Graduates (RCG) 

98-15 Development of a Prototype System for Accessing Linked NCES Data 

Schools and Staffing Survey (SASS) 

94-0 1 Schools and Staffing Survey (SASS) Papers Presented at Meetings of the American 
Statistical Association 

94-02 Generalized Variance Estimate for Schools and Staffing Survey (SASS) 

94-03 1 99 1 Schools and Staffing Survey (SASS) Reinterview Response Variance Report 

94-04 The Accuracy ofTeachers’ Self-reports on their Postsecondary Education: Teacher 

Transcript Study, Schools and Staffing Survey 

94- 06 Six Papers on Teachers from the 1 990-91 Schools and Staffing Survey and Other Related 

Surveys 

95- 01 Schools and Staffing Survey: 1994 Papers Presented at the 1994 Meeting of the American 

Statistical Association 

95-02 QED Estimates of the 1 990-91 Schools and Staffing Survey: Deriving and Comparing 
QED School Estimates with CCD Estimates 
95-03 Schools and Staffing Survey: 1990-91 SASS Cross-Questionnaire Analysis 
95-08 CCD Adjustment to the 1990-91 SASS: A Comparison of Estimates 
95-09 The Results of the 1 993 Teacher List Validation Study (TLVS) 

95-10 The Results of the 1 991-92 Teacher Follow-up Survey (TFS) Reinterview and Extensive 

Reconciliation 




Kathryn Chandler 
Kathryn Chandler 
Peter Stowe 
Peter Stowe 



Samuel Peng 



Andrew G. Malizio 



Linda Zimbler 
Steven Kaufman 
Linda Zimbler 



Aurora D’Amico 



Steven Kaufman 
Stephen Broughman 
Stephen Broughman 
Steven Kaufman 
Steven Kaufman 
Stephen Broughman 

Stephen Broughman 
Steven Kaufman 
Dan Kasprzyk 

Stephen Broughman 



Steven Kaufman 



Dan Kasprzyk 

Dan Kasprzyk 
Dan Kasprzyk 
Dan Kasprzyk 

Dan Kasprzyk 

Dan Kasprzyk 

Dan Kasprzyk 

Dan Kasprzyk 
Dan Kasprzyk 
Dan Kasprzyk 
Dan Kasprzyk 
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No. 



Title 



NCES contact 



95-11 

95-12 

95-14 

95-15 

95-16 

95- 18 

96- 01 

96-02 

96-05 

96-06 

96-07 

96-09 

96-10 

96-11 

96-12 

96-15 

96-23 

96-24 

96-25 

96- 28 

97- 01 

97-07 

97-09 

97-10 

97-11 

97-12 

97-14 

97-18 

97-22 

97-23 

97-41 

97-42 

97- 44 

98- 01 
98-02 
98-04 
98-05 

98-08 

98-12 



Measuring Instruction, Curriculum Content, and Instructional Resources: The Status of 
Recent Work 

Rural Education Data User’s Guide 

Empirical Evaluation of Social, Psychological, & Educational Construct Variables Used 
in NCES Surveys 

Classroom Instructional Processes: A Review of Existing Measurement Approaches and 
Their Applicability for the Teacher Follow-up Survey 
Intersurvey Consistency in NCES Private School Surveys 

An Agenda for Research on Teachers and Schools: Revisiting NCES’ Schools and 
Staffing Survey 

Methodological Issues in the Study of Teachers’ Careers: Critical Features of a Truly 
Longitudinal Study 

Schools and Staffing Survey (SASS): 1995 Selected papers presented at the 1995 Meeting 
of the American Statistical Association 

Cognitive Research on the Teacher Listing Form for the Schools and Staffing Survey 
The Schools and Staffing Survey (SASS) for 1998-99: Design Recommendations to 
Inform Broad Education Policy 

Should SASS Measure Instructional Processes and Teacher Effectiveness? 

Making Data Relevant for Policy Discussions: Redesigning the School Administrator 
Questionnaire for the 1998-99 SASS 
1998-99 Schools and Staffing Survey: Issues Related to Survey Depth 
Towards an Organizational Database on America’s Schools: A Proposal for the Future of 
SASS, with comments on School Reform, Governance, and Finance 
Predictors of Retention, Transfer, and Attrition of Special and General Education 
Teachers: Data from the 1989 Teacher Followup Survey 
Nested Structures: District-Level Data in the Schools and Staffing Survey 
Linking Student Data to SASS: Why, When, How 
National Assessments of Teacher Quality 

Measures of Inservice Professional Development: Suggested Items for the 1998-1999 
Schools and Staffing Survey 

Student Learning, Teaching Quality, and Professional Development: Theoretical 
Linkages, Current Measurement, and Recommendations for Future Data Collection 
Selected Papers on Education Surveys: Papers Presented at the 1 996 Meeting of the 
American Statistical Association 

The Determinants of Per-Pupil Expenditures in Private Elementary and Secondary 
Schools: An Exploratory Analysis 
Status of Data on Crime and Violence in Schools: Final Report 
Report of Cognitive Research on the Public and Private School Teacher Questionnaires 
for the Schools and Staffing Survey 1993-94 School Year 
International Comparisons of Inservice Professional Development 
Measuring School Reform: Recommendations for Future SASS Data Collection 
Optimal Choice of Periodicities for the Schools and Staffing Survey: Modeling and 
Analysis 

Improving the Mail Return Rates of SASS Surveys: A Review of the Literature 
Collection of Private School Finance Data: Development of a Questionnaire 
Further Cognitive Research on the Schools and Staffing Survey (SASS) Teacher Listing 
Form 

Selected Papers on the Schools and Staffing Survey: Papers Presented at the 1 997 Meeting 
of the American Statistical Association 

Improving the Measurement of Staffing Resources at the School Level: The Development 
of Recommendations for NCES for the Schools and Staffing Survey (SASS) 
Development of a SASS 1993-94 School-Level Student Achievement Subfile: Using 
State Assessments and State NAEP, Feasibility Study 
Collection of Public School Expenditure Data: Development of a Questionnaire 
Response Variance in the 1993-94 Schools and Staffing Survey: A Reinterview Report 
Geographic Variations in Public Schools’ Costs 

SASS Documentation: 1993-94 SASS Student Sampling Problems; Solutions for 

Determining the Numerators for the SASS Private School (3B) Second-Stage Factors 
The Redesign of the Schools and Staffing Survey for 1999-2000: A Position Paper 
A Bootstrap Variance Estimator for Systematic PPS Sampling 



Sharon Bobbitt & 
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Samuel Peng 
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Mary Rollefson 

Dan Kasprzyk 

Stephen Broughman 

Lee Hoffman 
Dan Kasprzyk 

Dan Kasprzyk 
Mary Rollefson 
Steven Kaufman 

Steven Kaufman 
Stephen Broughman 
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Stephen Broughman 
Steven Kaufman 
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Steven Kaufman 
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98-13 

98-14 

98-15 

98-16 

1999-02 

1999-04 

1999-07 

1999-08 

1999-10 

1999-12 

1999-13 

1999-14 

1999 - 17 

2000- 04 

2000-10 

2000-13 



0 




Response Variance in the 1994-95 Teacher Follow-up Survey 
Variance Estimation of Imputed Survey Data 

Development of a Prototype System for Accessing Linked NCES Data 
A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 
Tracking Secondary Use of the Schools and Staffing Survey Data: Preliminary Results 
Measuring Teacher Qualifications 

Collection of Resource and Expenditure Data on the Schools and Staffing Survey 
Measuring Classroom Instructional Processes: Using Survey and Case Study Fieldtest 
Results to Improve Item Construction 
What Users Say About Schools and Staffing Survey Publications 
1993-94 Schools and Staffing Survey: Data File User’s Manual, Volume III: Public-Use 
Co debook 

1993- 94 Schools and Staffing Survey: Data File User’s Manual, Volume IV: Bureau of 
Indian Affairs (BIA) Restricted-Use Codebook 

1994- 95 Teacher Followup Survey: Data File User’s Manual, Restricted-Use Codebook 
Secondary Use of the Schools and Staffing Survey Data 

Selected Papers on Education Surveys: Papers Presented at the 1998 and 1999 ASA and 
1999 AAPOR Meetings 

A Research Agenda for the 1 999-2000 Schools and Staffing Survey 
Non-professional Staff in the Schools and Staffing Survey (SASS) and Common Core of 
Data (CCD) 




Steven Kaufman 
Steven Kaufman 
Steven Kaufman 
Stephen Brough man 
Dan Kasprzyk 
Dan Kasprzyk 
Stephen Brough man 
Dan Kasprzyk 

Dan Kasprzyk 
Kerry Gruber 

Kerry Gruber 

Kerry Gruber 
Susan Wiley 
Dan Kasprzyk 

Dan Kasprzyk 
Kerry Gruber 



Listing of NCES Working Papers by Subject 



No. Title 



NCES contact 



Adult education 

96-14 The 1 995 National Household Education Survey: Reinterview Results for the Adult 
Education Component 

96-20 1991 National Household Education Survey (NHES:91) Questionnaires: Screener, Early 

Childhood Education, and Adult Education 

96- 22 1995 National Household Education Survey (NHES:95) Questionnaires: Screener, Early 

Childhood Program Participation, and Adult Education 
98-03 Adult Education in the 1990s: A Report on the 1991 National Household Education 
Survey 

98-10 Adult Education Participation Decisions and Barriers: Review of Conceptual Frameworks 
and Empirical Studies 

1 999- 1 1 Data Sources on Lifelong Learning Available from the National Center for Education 

Statistics 

2000-1 6a Lifelong Learning NCES Task Force: Final Report Volume I 

2000-1 6b Lifelong Learning NCES Task Force: Final Report Volume II 

Adult literacy — see Literacy of adults 

American Indian - education 

1999-13 1993-94 Schools and Staffing Survey: Data File User’s Manual, Volume IV: Bureau of 

Indian Affairs (BIA) Restricted-Use Codebook 

Assessment/achievement 
95-12 Rural Education Data User’s Guide 

95-13 Assessing Students with Disabilities and Limited English Proficiency 

97- 29 Can State Assessment Data be Used to Reduce State NAEP Sample Sizes? 

97-30 ACT’s NAEP Redesign Project: Assessment Design is the Key to Useful and Stable 
Assessment Results 

97-31 NAEP Reconfigured: An Integrated Redesign of the National Assessment of Educational 
Progress 

97-32 Innovative Solutions to Intractable Large Scale Assessment (Problem 2: Background 
Questions) 

97-37 Optimal Rating Procedures and Methodology for NAEP Open-ended Items 

97- 44 Development of a SASS 1993-94 School-Level Student Achievement Subfile: Using 

State Assessments and State NAEP, Feasibility Study 

98- 09 High School Curriculum Structure: Effects on Coursetaking and Achievement in 

Mathematics for High School Graduates — An Examination of Data from the National 
Education Longitudinal Study of 1988 

Beginning students in postsecondary education 

98-1 1 Beginning Postsecondary Students Longitudinal Study First Follow-up (BPS:96-98) Field 
Test Report 

Civic participation 

97-25 1 996 National Household Education Survey (NHES:96) Questionnaires: 

Screener/Household and Library, Parent and Family Involvement in Education and 
Civic Involvement, Youth Civic Involvement, and Adult Civic Involvement 

Climate of schools 

95-14 Empirical Evaluation of Social, Psychological, & Educational Construct Variables Used 
in NCES Surveys 

Cost of education indices 

94-05 Cost-of-Education Differentials Across the States 
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Course-taking 

95-12 Rural Education Data User’s Guide 

98-09 High School Curriculum Structure: Effects on Coursetaking and Achievement in 

Mathematics for High School Graduates — An Examination of Data from the National 
Education Longitudinal Study of 1988 
1999-05 Procedures Guide for Transcript Studies 
1 999-06 1 998 Revision of the Secondary School Taxonomy 

Crime 

97- 09 Status of Data on Crime and Violence in Schools: Final Report 

Curriculum 

95-1 1 Measuring Instruction, Curriculum Content, and Instructional Resources: The Status of 
Recent Work 

98- 09 High School Curriculum Structure: Effects on Coursetaking and Achievement in 

Mathematics for High School Graduates — An Examination of Data from the National 
Education Longitudinal Study of 1988 

Customer service 

1999- 10 What Users Say About Schools and Staffing Survey Publications 

2000- 02 Coordinating NCES Surveys: Options, Issues, Challenges, and Next Steps 

2000-04 Selected Papers on Education Surveys: Papers Presented at the 1998 and 1999 ASA and 

1999 AAPOR Meetings 



Data quality 

97-13 Improving Data Quality in NCES: Database-to-Report Process 
Data warehouse 

2000-04 Selected Papers on Education Surveys: Papers Presented at the 1998 and 1999 ASA and 
1999 AAPOR Meetings 



Design effects 

2000-03 Strengths and Limitations of Using SUDAAN, Stata, and WesVarPC for Computing 
Variances from NCES Data Sets 

Dropout rates, high school 

95- 07 National Education Longitudinal Study of 1988: Conducting Trend Analyses HS&B and 

NELS:88 Sophomore Cohort Dropouts 

Early childhood education 

96- 20 1991 National Household Education Survey (NHES:91) Questionnaires: Screener, Early 

Childhood Education, and Adult Education 

96- 22 1995 National Household Education Survey (NHES:95) Questionnaires: Screener, Early 

Childhood Program Participation, and Adult Education 

97- 24 Formulating a Design for the ECLS: A Review of Longitudinal Studies 

97- 36 Measuring the Quality of Program Environments in Head Start and Other Early Childhood 

Programs: A Review and Recommendations for Future Research 

1 999- 0 1 A Birth Cohort Study: Conceptual and Design Considerations and Rationale 

Educational attainment 

98- 1 1 Beginning Postsecondary Students Longitudinal Study First Follow-up (BPS:96-98) Field 

Test Report 

Educational research 

2000- 02 Coordinating NCES Surveys: Options, Issues, Challenges, and Next Steps 

Employment 

96-03 National Education Longitudinal Study of 1 988 (NELS:88) Research Framework and 
Issues 
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98-1 1 Beginning Postsecondary Students Longitudinal Study First Follow-up (BPS:96-98) Field 
Test Report 

2000-1 6a Lifelong Learning NCES Task Force: Final Report Volume I 

2000-1 6b Lifelong Learning NCES Task Force: Final Report Volume II 

Engineering 

2000-1 1 Financial Aid Profile of Graduate Students in Science and Engineering 

Faculty - higher education 

97- 26 Strategies for Improving Accuracy of Postsecondary Faculty Lists 

2000-0 1 1 999 National Study of Postsecondary Faculty (NSOPF:99) Field Test Report 

Finance - elementary and secondary schools 

94- 05 Cost-of-Education Differentials Across the States 

96- 1 9 Assessment and Analysis of School-Level Expenditures 

98- 01 Collection of Public School Expenditure Data: Development of a Questionnaire 

1999-07 Collection of Resource and Expenditure Data on the Schools and Staffing Survey 

1 999- 16 Measuring Resources in Education: From Accounting to the Resource Cost Model 

Approach 

Finance - postsecondary 

97- 27 Pilot Test of IPEDS Finance Survey 

2000- 14 IPEDS Finance Data Comparisons Under the 1997 Financial Accounting Standards for 

Private, Not-for-Profit Institutes: A Concept Paper 

Finance - private schools 

95- 17 Estimates of Expenditures for Private K-12 Schools 

96- 1 6 Strategies for Collecting Finance Data from Private Schools 

97- 07 The Determinants of Per-Pupil Expenditures in Private Elementary and Secondary 

Schools: An Exploratory Analysis 

97- 22 Collection of Private School Finance Data: Development of a Questionnaire 

1999- 07 Collection of Resource and Expenditure Data on the Schools and Staffing Survey 

2000- 15 Feasibility Report: School-Level Finance Pretest, Private School Questionnaire 

Geography 

98- 04 Geographic Variations in Public Schools’ Costs 

Graduate students 

2000-1 1 Financial Aid Profile of Graduate Students in Science and Engineering 

Imputation 

2000-04 Selected Papers on Education Surveys: Papers Presented at the 1998 and 1999 ASA and 
1999 AAPOR Meetings 

Inflation 

97-43 Measuring Inflation in Public School Costs 

Institution data 

2000-01 1999 National Study of Postsecondary Faculty (NSOPF:99) Field Test Report 

Instructional resources and practices 

95-1 1 Measuring Instruction, Curriculum Content, and Instructional Resources: The Status of 
Recent Work 

1 999-08 Measuring Classroom Instructional Processes: Using Survey and Case Study Field Test 
Results to Improve Item Construction 

International comparisons 

97-1 1 International Comparisons of Inservice Professional Development 

97-16 International Education Expenditure Comparability Study: Final Report, Volume I 




Aurora D’Amico 

Lisa Hudson 
Lisa Hudson 



Aurora D’Amico 



Linda Zimbler 
Linda Zimbler 



William J. Fowler, Jr. 
William J. Fowler, Jr. 
Stephen Broughman 
Stephen Broughman 
William J. Fowler, Jr. 



Peter Stowe 
Peter Stowe 



Stephen Broughman 
Stephen Broughman 
Stephen Broughman 

Stephen Broughman 
Stephen Broughman 
Stephen Broughman 



William J. Fowler, Jr. 



Aurora D’Amico 



Dan Kasprzyk 



William J. Fowler, Jr. 



Linda Zimbler 



Sharon Bobbitt & 
John Ralph 
Dan Kasprzyk 



Dan Kasprzyk 
Shelley Bums 



57 



No. 



Title 



NCES contact 



97- 17 International Education Expenditure Comparability Study: Final Report, Volume II, 

Quantitative Analysis of Expenditure Comparability 

Data Comparability and Public Policy: New Interest in Public Library Data Papers 
Presented at Meetings of the American Statistical Association 
1996 National Household Education Survey (NHES:96) Questionnaires: 

Screener/Household and Library, Parent and Family Involvement in Education and 
Civic Involvement, Youth Civic Involvement, and Adult Civic Involvement 

Limited English Proficiency 

95- 13 Assessing Students with Disabilities and Limited English Proficiency 

Literacy of adults 

98- 17 Developing the National Assessment of Adult Literacy: Recommendations from 

Stakeholders 

1999-09a 1992 National Adult Literacy Survey: An Overview 

1999-09b 1992 National Adult Literacy Survey: Sample Design 

1999-09c 1992 National Adult Literacy Survey: Weighting and Population Estimates 

1 999-09d 1992 National Adult Literacy Survey: Development of the Survey Instruments 

1999-09e 1992 National Adult Literacy Survey: Scaling and Proficiency Estimates 

1999-09f 1992 National Adult Literacy Survey: Interpreting the Adult Literacy Scales and Literacy 

Levels 

1999-09g 1992 National Adult Literacy Survey: Literacy Levels and the Response Probability 

Convention 

1 999- 1 1 Data Sources on Lifelong Learning Available from the National Center for Education 

Statistics 

2000- 05 Secondary Statistical Modeling With the National Assessment of Adult Literacy: 

Implications for the Design of the Background Questionnaire 
2000-06 Using Telephone and Mail Surveys as a Supplement or Alternative to Door-to-Door 
Surveys in the Assessment of Adult Literacy 

2000-07 “How Much Literacy is Enough?” Issues in Defining and Reporting Performance 
Standards for the National Assessment of Adult Literacy 
2000-08 Evaluation of the 1992 NALS Background Survey Questionnaire: An Analysis of Uses 
with Recommendations for Revisions 
2000-09 Demographic Changes and Literacy Development in a Decade 

Literacy of adults - international 

97- 33 Adult Literacy: An International Perspective 

Mathematics 

98- 09 High School Curriculum Structure: Effects on Coursetaking and Achievement in 

Mathematics for High School Graduates — An Examination of Data from the National 
Education Longitudinal Study of 1988 

1999-08 Measuring Classroom Instructional Processes: Using Survey and Case Study Field Test 
Results to Improve Item Construction 

Parental involvement in education 

96- 03 National Education Longitudinal Study of 1988 (NELS:88) Research Framework and 

Issues 

97- 25 1 996 National Household Education Survey (NHES:96) Questionnaires: 

Screener/Household and Library, Parent and Family Involvement in Education and 
Civic Involvement, Youth Civic Involvement, and Adult Civic Involvement 
1999-01 A Birth Cohort Study: Conceptual and Design Considerations and Rationale 

Participation rates 

98- 1 0 Adult Education Participation Decisions and Barriers: Review of Conceptual Frameworks 

and Empirical Studies 
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Postsecondary education 

1 999- 1 1 Data Sources on Lifelong Learning Available from the National Center for Education 

Statistics 

2000- 1 6a Lifelong Learning NCES Task Force: Final Report Volume I 

2000-1 6b Lifelong Learning NCES Task Force: Final Report Volume II 

Postsecondary education - persistence and attainment 

98-1 1 Beginning Postsecondary Students Longitudinal Study First Follow-up (BPS:96-98) Field 
Test Report 

1999- 15 Projected Postsecondary Outcomes of 1992 High School Graduates 

Postsecondary education - staff 

97-26 Strategies for Improving Accuracy of Postsecondary Faculty Lists 

2000- 01 1999 National Study of Postsecondary Faculty (NSOPF:99) Field Test Report 

Principals 

2000-10 A Research Agenda for the 1999-2000 Schools and Staffing Survey 
Private schools 

96- 16 Strategies for Collecting Finance Data from Private Schools 

97- 07 The Determinants of Per-Pupil Expenditures in Private Elementary and Secondary 

Schools: An Exploratory Analysis 

97-22 Collection of Private School Finance Data: Development of a Questionnaire 
2000-13 Non-professional Staff in the Schools and Staffing Survey (SASS) and Common Core of 
Data (CCD) 

2000-1 5 Feasibility Report: School-Level Finance Pretest, Private School Questionnaire 

Projections of education statistics 

1 999- 1 5 Projected Postsecondary Outcomes of 1 992 High School Graduates 

Public school finance 

1999-16 Measuring Resources in Education: From Accounting to the Resource Cost Model 
Approach 



Public schools 

97- 43 Measuring Inflation in Public School Costs 

98- 01 Collection of Public School Expenditure Data: Development of a Questionnaire 

98-04 Geographic Variations in Public Schools’ Costs 

1 999- 02 Tracking Secondary Use of the Schools and Staffing Survey Data: Preliminary Results 

2000- 12 Coverage Evaluation of the 1994-95 Public Elementary/Secondary School Universe 

Survey 

2000-13 Non-professional Staff in the Schools and Staffing Survey (SASS) and Common Core of 
Data (CCD) 

Public schools - secondary 

98-09 High School Curriculum Structure: Effects on Coursetaking and Achievement in 

Mathematics for High School Graduates — An Examination of Data from the National 
Education Longitudinal Study of 1 988 

Reform, educational 

96-03 National Education Longitudinal Study of 1 988 (NELS:88) Research Framework and 
Issues 



Response rates 

98-02 Response Variance in the 1993-94 Schools and Staffing Survey: A Reinterview Report 

School districts 

2000- 1 0 A Research Agenda for the 1999-2000 Schools and Staffing Survey 
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School districts, public 

98-07 Decennial Census School District Project Planning Report 

1 999-03 Evaluation of the 1 996-97 Nonfiscal Common Core of Data Surveys Data Collection, 
Processing, and Editing Cycle 

School districts, public - demographics of 

96- 04 Census Mapping Project/School District Data Book 

Schools 

97- 42 Improving the Measurement of Staffing Resources at the School Level: The Development 

of Recommendations for NCES for the Schools and Staffing Survey (SASS) 

98- 08 The Redesign of the Schools and Staffing Survey for 1999-2000: A Position Paper 

1 999- 03 Evaluation of the 1996-97 Nonfiscal Common Core of Data Surveys Data Collection, 

Processing, and Editing Cycle 

2000- 10 A Research Agenda for the 1 999-2000 Schools and Staffing Survey 

Schools - safety and discipline 

97-09 Status of Data on Crime and Violence in Schools: Final Report 

Science 

2000-1 1 Financial Aid Profile of Graduate Students in Science and Engineering 

Software evaluation 

2000-03 Strengths and Limitations of Using SUDAAN, Stata, and WesVarPC for Computing 
Variances from NCES Data Sets 

Staff 

97- 42 Improving the Measurement of Staffing Resources at the School Level: The Development 

of Recommendations for NCES for the Schools and Staffing Survey (SASS) 

98- 08 The Redesign of the Schools and Staffing Survey for 1999-2000: A Position Paper 

Staff - higher education institutions 

97-26 Strategies for Improving Accuracy of Postsecondary Faculty Lists 

Staff - nonprofessional 

2000-13 Non-professional Staff in the Schools and Staffing Survey (SASS) and Common Core of 
Data (CCD) 

State 

1999-03 Evaluation of the 1996-97 Nonfiscal Common Core of Data Surveys Data Collection, 
Processing, and Editing Cycle 

Statistical methodology 

97-21 Statistics for Policymakers or Everything You Wanted to Know About Statistics But 
Thought You Could Never Understand 

Students with disabilities 

95- 13 Assessing Students with Disabilities and Limited English Proficiency 

Survey methodology 

96- 1 7 National Postsecondary Student Aid Study: 1 996 Field Test Methodology Report 

97- 1 5 Customer Service Survey: Common Core of Data Coordinators 

97- 35 Design, Data Collection, Interview Administration Time, and Data Editing in the 1996 

National Household Education Survey 

98- 06 National Education Longitudinal Study of 1988 (NELS:88) Base Year through Second 

Follow-Up: Final Methodology Report 

98-1 1 Beginning Postsecondary Students Longitudinal Study First Follow-up (BPS:96-98) Field 
Test Report 

98-16 A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 
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1999-07 


Collection of Resource and Expenditure Data on the Schools and Staffing Survey 


Stephen Brough man 


1999-17 


Secondary Use of the Schools and Staffing Survey Data 


Susan Wiley 


2000-01 


1999 National Study of Postsecondary Faculty (NSOPF:99) Field Test Report 


Linda Zimbler 


2000-02 


Coordinating NCES Surveys: Options, Issues, Challenges, and Next Steps 


Valena Plisko 


2000-04 


Selected Papers on Education Surveys: Papers Presented at the 1998 and 1 999 ASA and 


Dan Kasprzyk 




1999 AAPOR Meetings 




2000-12 


Coverage Evaluation of the 1994-95 Public Elementary/Secondary School Universe 


Beth Young 




Survey 




Teachers 






98-13 


Response Variance in the 1994-95 Teacher Follow-up Survey 


Steven Kaufman 


1999-14 


1994-95 Teacher Followup Survey: Data File User’s Manual, Restricted -Use Codebook 


Kerry Gruber 


2000-10 


A Research Agenda for the 1999-2000 Schools and Staffing Survey 


Dan Kasprzyk 


Teachers 


- instructional practices of 




98-08 


The Redesign of the Schools and Staffing Survey for 1999-2000: A Position Paper 


Dan Kasprzyk 


Teachers 


- opinions regarding safety 




98-08 


The Redesign of the Schools and Staffing Survey for 1999-2000: A Position Paper 


Dan Kasprzyk 


Teachers 


- performance evaluations 




1999-04 


Measuring Teacher Qualifications 


Dan Kasprzyk 


Teachers 


- qualifications of 




1999-04 


Measuring Teacher Qualifications 


Dan Kasprzyk 


Teachers 


- salaries of 




94-05 


Cost-of-Education Differentials Across the States 


William J. Fowler, Jr. 


Training 






2000-1 6a 


Lifelong Learning NCES Task Force: Final Report Volume I 


Lisa Hudson 


2000-1 6b 


Lifelong Learning NCES Task Force: Final Report Volume II 


Lisa Hudson 


Variance 


estimation 




2000-03 


Strengths and Limitations of Using SUDAAN, Stata, and WesVarPC for Computing 
Variances from NCES Data Sets 


Ralph Lee 


2000-04 


Selected Papers on Education Surveys: Papers Presented at the 1998 and 1999 ASA and 


Dan Kasprzyk 




1999 AAPOR Meetings 




Violence 






97-09 


Status of Data on Crime and Violence in Schools: Final Report 


Lee Hoffman 


Vocational education 




95-12 


Rural Education Data User’s Guide 


Samuel Peng 


1999-05 


Procedures Guide for Transcript Studies 


Dawn Nelson 


1999-06 


1998 Revision of the Secondary School Taxonomy 


Dawn Nelson 
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